I like the idea that people are tested and judged by the paintings and the music they think they are evaluating. And in day to day life, if I find someone sneers at something that I value and admire greatly, there are only two things that can happen. Either I consider whether I'm wrong about the work in question, or my opinion of that person goes down. Similarly when they admire something I think is crap, either my opinion of them goes down, or I have another look at what they are admiring.
I can't see those kind of judgements as irrelevant. The crucial thing is how much respect I have for that person's judgement. (And I tend to have very little respect indeed for professional critics in any field.)
Everyone's got a right to think what they like, and tell the world. And everyone else had the right to think that what they are saying is total rubbish.