Cecil Sharp collected The Orange Tree from William Pittaway at Burford, Oxfordshire, in 1923. Unfortunately, Sharp only noted the tune and the first verse - and I cannot find any other versions anywhere. The first verse goes as follows:

'Twas in the month of May when flowers they were gay,
when the orange trees were all in bloom,
when the orange orange orange trees were all in bloom'.

Does anyone have the rest?