The running joke in this song is that the singer always avoids saying "bum"—the expected word according to the rhyme scheme—by instead singing the nonsense chorus, "Bumpsy toralee...."

The way it's printed at Reinhard Zierke's Watersons page makes this a bit more clear:

Why, her father catched me courting her and after me he run
And he says, "If I catch thee with her again I shall whack thee round the ..."
Bumpsy tooralee me titty falloora lee me,
Rumpsy bumpsy diddledee umpsy rumpsy tooralee.

There are many other songs that use the same device, e.g. the kids' song Miss Susie [or Lucy] Had a Steamboat.

Wouldn't it spoil the joke to use the even stronger word "arse" elsewhere in the same song? (At least it seems stronger to me; what would a native British-English speaker say?)