I suppose people would feel just as cheerful about the idea of some young lout doing the same ham sandwich bit to a Jewish child refugee back in the 30s.

Actually they probably would if they'd been around at that time, I suspect. The target may vary, but the mindset remains the same.