It's theft (as a matter of UK law) - probably, although there is an issue about intent permanently to deprive.

I don't immediately see why you should not turn his amplifier off (so long as you plan to enjoy the fight when he emerges bent on killing you for the insult).