It would count as derivative work unless the source material is over 100 years old, in which case it would be in the public domain and free to use. Generally, yes, that is copyright infringement, but also pretty much every con has artists selling fan art and some of them make a decent amount of money on it. It's possible you could get sued or slapped with a cease and desist, though the odds are fairly low. Sucks if it happens though.

Re: the first question, in my contracts I make it clear that I own the copyright to work created and they are paying for the creation and licensing of the work for use in whatever project. Usually you give a time frame, ex. the interior of the book and any associated promotional material regarding the book printed over the next 2 years.

After that they would need to pay an additional licensing fee to use it again. That said, this is the sort of thing an agent would demand, lots of artists without agents who need the commission money can't be as picky. Generally it's a good idea to never sell the copyright to your work though - if a customer ever asks me for the copyright I charge triple and they usually change their minds. If you're working for a big company like Disney some of them have really strict ownership contracts, but in the freelance world that's a lot less common. Hope this helps!