Shambles, I refer you to the Bugle Call Rag.There are varying degrees of improvisation. In the jazz world, it's assumed that you know the tune and can play the melody and a decent set of chord changes before you take off on it. Style is also a factor. Be-bop doesn't work in a jazz band very well. It's a force fit.
Many great improvisers such as Mozart, Charlie Parker and Django Reinhardt did know what they were doing but not in a conventional sense of being able to substantiate it with theoretical explanations. They knew it by feel and experience having lived with the music they played for a long disciplined period of time. They knew their instrument having played and practiced. The quality of their improvisation is predicated on the last statement.
Frank