This is entirely from memory, so it may not be 100% accurate:

VICAR IS A BEATNIK
(Sydney Carter)

'Good morning', said the vicar, with a banjo round his neck;
'We're digging up the crypt', he said, 'to make a discotheque'.
So we're writing to the bishop to say that we are shocked,
For the vicar is a beatnik and he ought to be defrocked.

At early morning service he plays a mandolin;
I'm never there to hear it but I think it is a sin.
So we're writing to the bishop to say that we are shocked,
For the vicar is a beatnik and he ought to be defrocked.

I love the merry organ and the bells across the snow;
I love the Church of England, although I never go.
I love the dear old Bible with 'Jehovah', and 'begat' -
It's not that I believe in it or anything like that!
(It's not that we believe in it or anything like that).

Just marry us and bury us the way you used to do -
With a blast upon the organ, and not on your kazoo.
So we're writing to the bishop to say that we are shocked,
For the vicar is a beatnik and he ought to be defrocked.