Gareth, you mount a horse on the left side. If you are right handed your sword hangs on your left side. That way it is not in the way when you mount the horse.

I guess left handed sword fighters mounted on the right but since they were all sinister and, therefore, in league with the Devil what they did hasn't come down through history to us.