I'm left-handed and play guitars etc right-handed, the other way seems wrong. Piano, obviously there was never a choice. Harmonica I hold slightly unusually, but still the "right" way round.

It makes sense for me in all three cases that low notes are to the left, high to the right. In fact I find it looks quite odd seeing someone play the other way round (even on harmonica)

However, I've been learning drums, and I have to set the kit up the other way. If I try playing a right-handed drum kit I end up getting the kick and snare the wrong way round.

As a guitar teacher I've had a few left-handed pupils, most have learned right-handed by default, but I'd definitely advise anyone to try both ways to see what feels best.