I've had problems in the past, but I'm sorted now! (with USB connectors, that is)

The metal of the plug has a seam on the 'bottom'.

Every socket I've found is in that orientation. Oh, except those that are sideways. For these, the 'bottom' is almost always the bottom of the motherboard, and so the nearest side of the case.

So, look at the plug, hold it the right way, and Robert's your mother's brother.

Phil