What I think is most frowned on is "singing from THE book," especially when THE book is a commonly-available book and no OTHER book is considered "correct." That is, some song circles use Rise Up Singing or The Folksinger's Wordbook (an earlier version of same, I think), and they treat it like a hymnal. If you want to sing a version that is different from the book's, you'll get pushback, if not actual argument.
Now, singing from a memory-aid, to me is different. I don't always recall the songs I want to do, and have been known to use a print-out (or to "look it up on my fingers" during a break, to make sure I can recall the thing), and don't get too worked up if someone has a sheet for their own use.
Part of it is the group dynamic; I'm used to the FSGW open sing "manual," which says you don't chide someone for using a book, but we're resistant to following the book if the song is one of our well-known ones where we differ from the book. I haven't been to too many sings where a book -- any book -- is used as an "authority," but I have been to at least one or two. I usually don't go back.
Actually, I just remembered that with the Civil War Comrades, the reenactment trio I'm in, we often use lyric sheets onstage, to avoid the dreaded "version creep," where one of us knows a different word or two in the choruses, where we're supposed to be singing together.