"I think we all have the desire to be polite to that one individual who sings off key or songs we don't like or whatever. But then are 4 and then 7 and then 10 and in inverse proportion people leave."

You are describing ANY social group, it doesn't matter about the song. Insert "off key", "songs we don't like", or "the blue book" and you can find a reason to not wish to participate. Sure, these reasons may have some validity to a group - but again it comes back to the purpose of the group.   You say that they can set up "their own song circles or groups where they set it up exactly like they like it and others are free to join them or set up something else entirely again" - well, that is what happened to your group because no one wished to discuss it. The group was set up exactly the way they like it, and no one else raised their voice.

The "critical mass" IS the group.