I learned "Tha mi sgith" from the Kennedy-Fraser collection at about age 13 and have sung it ever since (I'm now 48), never having had any problem with the sex of the fairy person in question -- after all, they presumably come in both male and female versions, as do the people they lure away.

Bagpipers in this area (Ottawa, Ontario) play "Tha mi sgith" both as a dance tune (a hornpipe, I think) and as a quick march, and I recently met a fiddler who called it "Pulling Ferns" and had no notion it had lyrics at all.