When I first heard this song, I was fooled by the multiple rhyme & galloping rhythm into thinking it was Irish -- despite the repeated mention of Orkney & Dover, which eventually clued me in. Indeed, it & its author are 100% Scottish.
That, however, left a puzzle. My knowledge of the Christian religion is small, but somehow I have trouble imagining a Presbyterian saying "Jesus, at least I tried!". Does Mr Stewart perhaps belong to a Scottish *Catholic* family? (I once witnessed that phenomenon myself. Many years ago, while walking in -- I think it was -- Aberdeen with another young American tourist I had met at the station, we stopped in front of a brick edifice that the guidebook identified as a cathedral. I mused, "What does a cathedral mean in Scotland?" which brought a smile to the face of a priest who just happened to be passing.) Googling furiously, I found many mentions of a recent performance by him (at an Irish evening!) in a Catholic high school in Pittsburgh, PA.