Hero, damsel, singers, villain, doctor, etc. appear as well in Italian and other street and folk theatre, so the source need not be mumming. I have figurines of some the characters from Comedia della Arte (forgive my spelling, I won't bother to check).

We used to have a story (SW US and Mexico) of why the early Catholic missionaries so often were depicted holding a cross in front of them- it was to keep branches and spiderwebs from getting in their face. Now surely the natives would equate this with sweeping and would associate their besoms with the cross.