It's quite possible that the song was created solely to torture and humiliate Baptist young people (of which I was one). Back in the sixties you could not go to a camp or a picnic in my community without somebody whipping out an instrument and starting "Kum Ba Ya". Seriously, though, if it is a distortion of the English words "Come By Here" I'd bet that it started somewhere on this side of the Atlantic. Reminds me a lot of the Gullah dialect spoken in many communities here on the coast(s) of South Carolina and Georgia. And of course there are quite a few old hymns and spirituals that contain "Come by Here" or "Jesus Won't you come by Here."

Good luck, Chet W.