The improvisation involved in calypso is mainly verbal - the tunes pretty much follow the words. I imagine it involves continually spinning little fragments of poetry in your head, all the time, until they start to click together. I can't begin to do anything like that and I would go slightly insane trying to do it in isolation. Being surrounded by people who consider it okay and even admirable to do that would make a big difference.

The performance poetry scene might be a source of sympathetic ears, if it exists where you are.