I know it as Gathering Flowers. Slightly different words, but tune basically as per recording, not Here we come gathering nuts in May on a cold & frosty morning tune.

We were gathering flowers, flowers, flowers
Down by the old mill stream.
We were gathering flowers,
Billed and cooed for hours.
Down by the old mill stream.

No you can't have it tonight
For the moon's shining bright
And the stars are twinkling in the sky-hi-hi-eye
Everybody's watching us, watching you and I
So let's go round the corner and have it on the sly

We were gathering rosemary, rosemary, rosemary
Down by the old mill stream.
We were gathering rosemary,
Swore that he would marry me.
Down by the old mill stream.

No you can't have it tonight
For the moon's shining bright
And the stars are twinkling in the sky-hi-hi-eye
Everybody's watching us, watching you and I
So let's go round the corner and have it on the sly

We were gathering watercress, watercress, watercress
Down by the old mill stream.
We were gathering watercress,
Left me in a hell of a mess.
Down by the old mill stream.

No you can't have it tonight
For the moon's shining bright
And the stars are twinkling in the sky-hi-hi-eye
Everybody's watching us, watching you and I
So let's go round the corner and have it on the sly

We were gathering nuts in may, nuts in may, nuts in may
Down by the old mill stream.
We were gathering nuts in may,
Left me in the family way.
Down by the old mill stream.

No you can't have it tonight
For the moon's shining bright
And the stars are twinkling in the sky-hi-hi-eye
Everybody's watching us, watching you and I
So let's go round the corner and have it on the sly