Don't know anything about him being haunted, but here's a version from "The Folksinger's Wordbook" (Oak Publications), which I bought via the net.

Note the additional last verse, which is also on the Bert Jansch version (on Jack Orion).
[petejennings@supanet.com]

I courted Pretty Polly the live long night
And left her next morning before it was light.

Polly, pretty Polly, come go along with me
Before we get married, some pleasures to see.

She jumped up behind him and away they did go.
Over the hills and the valleys below.

They went a little further and what did they spy
But a new-dug grave with a spade lying by.

Oh Willie, oh Willie, I'm afraid of your ways
I'm afraid you will lead my poor body astray.

Polly, pretty Polly, you've guessed about right
I dug on your grave for the most of last night.

He throwed her on the ground and she broke into tears
She throwed her arms around him and trembled with fear.

There's no time to talk now, there's no time to stand
He drew out his knife all in his right hand.

He stabbed her in the heart and the blood it did flow
And into the grave pretty Polly did go.

He threw a little dirt over her and started for home
Leaving none back behind but the wild birds to moan

Now a debt to the Devil, Willie must pay
For killing pretty Polly and running away

HTML line breaks added --JoeClone, 29-Nov-01.