Smalls as a description probably came from the older expression "small clothes" ... how far back that goes I'm not sure, but a fair way, I think.
As to "toilette", its literal meaning is "small cloth" and was initially used to describe the cloth covering the chamber pot, or so I was told at some time in my youth. From there it became the pot itself, so euphemisms have always abounded.

I do remember some US/UK moments of incomprehension when I was working in Paris. Someone (American) asked where the mop was. I said it was in the toilet. A little while later the American brandished the mop at me and said how silly I'd been ... of course you wouldn't keep the mop in the toilet, as it would get wet. It had been in the bathroom, of course. Except from my point of view that would have been equally daft, as there was no bathroom. We had some showers, and some toilets, but no baths.