"Whom" is used far less in speech than in (certainly more formal styles of) writing. I'd far rather rejig the sentence than write "whom." I may be technically incorrect in saying this, but the only time I feel uneasy about using "who" instead is when a quantity word precedes it, for example "many of whom." It will fade away, unfortunately not before I do.