I agree, The Outside Track has an absolutely wonderful version of Caroline of Edinburgh Town. It's on their CD Curious Things Given Wings and has quickly become one of my favorite songs. As to the origin, they write. "Norah's (lead singer Norah Rendell) husband, Brian Miller, found this version in an unpublished archive of songs collected in the American midwest by Franz Rickaby. The source singer, Michael C Dean, said that his Irish immigrant mother used to sing the song to him as a lullaby." That could speak for the Irish origin, although it's odd that Irish would be singing about Edinburgh, unless they were Scots-Irish.

Among other songs about Edinburgh, there is also Edinburgh Toon. I have only heard it sung by North Sea Gas - on the CD of the same name. I am sure there are plenty of other versions. It's the same theme as the Black Velvet Band, about an unsuspecting young man from the country who goes to the city and falls prey to the wiles of a thieving young woman. He helps her with her robberies and himself ends up getting arrested.