Yes, some people can be "difficult" to listen to :>)
I find that on those occasions you can still enjoy their performance by listening to the song rather than the singing. At the very least you can appreciate their intentions - and they are likely to get better with experience rather than worse, so that can be very nice to hear as well.
The above comments are made with an informal singaround type situation in mind - other situations may be different.
Plus I'm in a nice mellow mood whilst writing it...... I expect I am sometimes guilty of dismissing someone's singing.....

Another point I would like to make is that sometimes its not so much that someone has said that one can't sing, as that no-one has said that one can. I didn't realise I could sing worth listening to until I was 34 - before that I knew I loved to sing, but just never realised anyone might like to hear it! So now I've turned into this monster woman who nags at her friends & family to just come along & have a go - and then I get all the fun of saying "I told you so"!