When NOT to sing - when what YOU are singing differs from what the performer is singing.

Not only different words or tune but different timing or singing a harmony can throw off a performer.

And if you are one of those performers who hates people singing along the answer is very simple. Sing something that they don't know.