I would too.
I think, from an outsider point of view, it would be more difficult male to female, if only because you can't undo the deepening of the voice.
I had a friend for at least 5 years, before he said he was trans. I never would've even thought about it.