Some linguist did a study of it recently. It contains Yiddish, eighteenth-century thieves cant, Cockney slang, and Roma dialect as well (or some amazing mixture like that). Saw an article about it in the Guardian.

I think Kenneth Williams objected because he didn't have much choice at the time --there were no real dramatic roles for a bloke like him and he felt (I believe) that the gay stereotypes perpetuated by Julian and Sandy contributed to the lack of public appreciation of gay issues at the time (when you laugh at a group, you don't have to take them seriously enough to register their concerns.) And it was still quite a homophobic time in Britain.

Others will argue that Julian and Sandy did a lot to increase tolerance. Maybe they did. But not enough to prevent Kenneth Williams feeling isolated, and you can understand why having to turn the thing on the basis of which he was discriminated against into a joke weighed on him over time. He didn't write the scripts, as the saying goes...