Singing with an accompaniment is different from singing without one. It's not "precious" to prefer to sing without an accompaniment when you know that this particular case an accompaniment will make it harder for you to sing the song the way you feel you can sing it best.

And that's not just when an accompanist isn't skilled. A brilliant accompanist can get in the way of a singer.

And the same can apply to joining in with other instruments when a singer is accompanying themselves - if it's a song you're breaking in it can really make you completely lose it to have someone come. Especially of course if they assume that the tune and the timing is slightly different from what it actually is intended to be.

That doesn't mean joining in is wrong - much of the time it's just what is wanted. But the basic rule must always be, if the person singing or playing indicates they don't want it, don't do it. And don't assume it's vanity on their part, it's much more likely to be the opposite.