"My response to this comment was - why not? - it's his really and he should be allowed to wear it if he wants to promote its true meaning."

Theoretically, you're right. Practically, depending on where he wears it, people might see it differnently and respond to him accordingly.

I still maintain that the response and connotation would be very different if it were a German wearing it, even if they had precisely the same intention. And to be honest, what would they really be achieving? As I said before, it's just a symbol. And maybe it's not a bad thing to have some symbolds that have been so comprehensively perverted that they stand as a warning to us never to repeat certain mistakes.