I am in possession of a copy of the broadside mentioned above by GUEST, currently in the possession of the New York Historical Society. This is the earliest known publication of this song that I can find, and from the way it's printed, as the foremost song among three on the page, with no author given, one gets the impression that the song was well known by the time of this (1820s) publication.

Here are the lyrics as printed on that broadside:

Peggy Gordon

Sweet Peggy Gordon, you are my darling,
Come set you down upon my knee,
And tell to me the very reason,
Why I am slighted so by thee.

I am deep in love, but I dare not show it,
My heart is lock'd up in thy breast;
I will plainly let the whole world know it,
A troubled mind can take no rest.

I'll lean my head on a cask of brandy,
That is my fancy I do declare;
For when I'm drinking, I'm always thinking,
How shall I gain that young lady fair.

I wish my love was one red rose,
And planted down by yonder wall,
And I myself was one drop of dew,
That in her bosom I might fall.

I wish my love and I were sailing,
As far from land as one could see;
Yes, sailing over the deepest waters,
Where love and care would not trouble me.

For the seas are deep, and I cannot wade them,
And neither have I wings to fly;
I wish I had some jolly boatman,
To ferry over my love and I.

I wish I was in Caropage,
And my sweet girl along with me;
Sweet Peggy Gordon, you are my darling;
Sweet Peggy Gordon, I'd die for thee.

I wish I was in Covel's Castle,
Where the marble stones are as black as ink,
Where the pretty girls they all adore me—
I'll sing no more until I drink.