John O'Lennaine's suggestion there about it being tied to Football (soccer) being (officially) non-contact, fits in with the fact that field games which are more physical on the pitch don't seem to be accompanied by crowd violence: Rugby, American Football, Gaelic Football and Hurling, Australian Rules Football, Ice Hockey...

On the other hand there isn't much crowd violence around Tennis matches either, and the same goes for Baseball and Cricket. So perhaps it's more complicated than that.