Bill and Gene Bonyun used to sing the chorus this way:

Oh, that I were where I would be,
Then I would be where I am not,
But I am where I must be,
Where I would be, I can not,
Oh, little lallie day, oh the little lee oh day.

Everyone else must have got it wrong! ;~)

Cheerily,
Charley Noble