I think any musicians, not just banjo players, need to be sensitive to what is going on when joining in.

If it's an unfamiliar song, or an unaccompanied singer has just kicked off in a very obscure key (I do it all the time I've been told), it can be very offputting. I find guitarists are usually OK as they tend to play very quietly until they pick it up, but at a recent session someone played a clarinet so loudly, with no empathy with the tune, that the singer had to stop and ask them (politely) to stop.

I'm also surprised at the number of amateur percussionists that don't seem to have a sense of rhythm!