By now we all realize that there is no one answer. I doubt that there is a consensus, so just play it by ear. If you can sing in public without a book, do so. If people leave it's probably because you can't sing.
If you need a book or you are ill prepared, or you suck as a singer people may leave. That should be your cue.
We all sing because we want to if we are at home or in a club. However, if we sing in public we should be singing for others as well as ourselves, and "Fuck 'em" is an attitude that will get you labeled "Persona non grata"