I remember it from school in the 1950s - we were not told what 'the rough game' it was associated with by our male teacher, the form master who did not take us for music.
I expect a later teacher, who taught us music and dancing might have been able to explain it - she always smelt of gin and was dismissed for falling off her bicycle in the playground as she arrived for a lesson. Just about every boy in the school claimed to have seen her underwear...