Perhaps one contributing factor to the confusion is our habit of shortening a phrase on an implied use. For instance, greengrocer's. I agree, of course, that it's not plural. However, it does require the apostrophe because of the unuttered, 'shop'.

Same as when I say that I stopped by (at) my parents'. I actually mean, my parents' home

Maybe that kind of thing causes confusion? I suspect that it's usually a case of never having paid actual attention. It doesn't bother me as it used to. Over the years I've gotten mellower about misusage.

For instance, I was taught that Bainbo's use of 'lying' is incorrect. One 'lays' something, therefore one would say 'laying down the rules'. On the other hand, one is 'lying down'.