Folks who are critical of Garrison's singing just don't get the concept of the show. Most of the music presented on the show is roots based music. I'm sure these same people would have to criticise Roscoe Holcomb, Doc Watson, Ralph Stanley or Karen Dalton if they had ever appeared on the show.
It brings to mind the Youtube comments on Pete Seeger's Rainbow Quest clips that say, "Why doesn't the dude with the banjo just shut up."
I'd be willing to bet that most of Garrison's guests enjoy singing with him.