Even aside from the small, but still real danger of infecting others (even in the case where we are properly vaccinated and recently tested), not wearing a mask in a crowded tube train is liable to make some people feel threatened, and it also serves to encourage others to choose to discard their marks. Anti-social behaviour is infectious.

If there is no threat then 'making someone feel threatened' is their problem, not yours.

Those who feel 'threatened' by what should be everyday occurrences can choose to keep themselves 'safe' at home.
Anyone who, genuinely, poses no danger to others, should not be prevented from going about their lawful business.
The fact that some people are over-cautious should be their problem, not someone elses.