mp -- I've been a singer for about 62 years. Started when I began to talk. I know what you mean. I noticed this when I was about five. AND boys could grow up to be explorers, spacemen, cowboys.
This is not to say that I'm unhappy -- and I wasn't then. Just to say I understand.
I've always sung some songs that don't gender-switch well -- but I'm very aware that it can be distracting. When I sing "Long Black Veil", for example, I get to "...been in the arms of my best friend's wife" and the image complicates things. Matters not that another soprano, and a famous one, sang it. For me, and for so many of us, songs have a life apart from the singer-of-the-moment.
Cheers Joy -- happily singing "I am a Young Man and I come from Kiandra" to myself.