Joe, some people seem to hear differences that I can see no reason for but there are other factors - here a couple I can think of:

The standard 12 TET scale equally spaces all the notes by a given interval and in some ways is a compromise scale, e.g. I think the 5th doen't sit where the natural 5th lies - the sort of thing where your ear may disagree with a tuner when tuning a guitar. Not all instruments are tuned this way, and therefore can give more noticible "feels" of different keys.

Stringed instruments sound different on the open strings to fretted strings.

Most intruments are simply easier to play in certain keys than others.

Jon