I think the term "Jew's harp" could be offensive because it implies that the stingy Jew would only invest in a harp of minute proportions. However, it seems more likely to me that the name is a corruption of "Jaws' harp"