Prejudice thrives in secrecy. If people don't talk about it, it doesn't exist in the 'normal' world and the possibility of outing a person can inspire fear. Obviously, the possibility of discussing controversial subjects can inspire fear, too.

A sex change/gender re-assignment is not something that will go unnoticed, if it does get talked about only in private.

From the number of people announcing how uncomfortable it makes them feel, I'd say the transformation AND those who've gone through it are taboo and folks would rather pretend they didn't exist.

Secrecy/taboos = power.