The situation I always used to found myself in was that depending on the time of day, the Key I would use or prefer, can be different each time, and musicians don't like that unless they are very very good!
There is also the fact that solo I sing in a more "conversational" style, and some musician again have problems following this. There are the need for more breaks in the action when you are accompanied, and that can sometime affect your rhythm and even the story.
If you are going to have musicians play with you it is best to really arrange and practice what you are going to do before hand - OR - If you are recording, add the music later!!
I know it can be classed as cheating, but sometimes it is necessary!!!

Tim Radford