Technically all drummers are percussionists but all percussionists are not drummers (so percussion is the overarching term). There appears to be less stigma attached to percussionists. There is an exception.

Multi-percussionists (eg: of classical, latin, Pink Floyd nature) as against single instrument percussionists (Bodhran, Djembe) tend to be stigmatised at festivals.

People hear with their eyes. Percussion is perceived to be loud. This is a stereotype. Percussion is the most dynamic instrument. Some players utilise the loud end too much.

Which gets me back to multi-percussionists (eg:drummers) and singing(or didge playing). Drummers who play and sing need to focus on their dynamics more. This is the case with didge as well.

This situation is even more pronounced in an acoustic setup. It is possible to play a drumkit/multi-percussion and sing at an acoustic level and have the vocals (or didge) be audible and intelligible.

Singing drummers, or didge playing drummers are rare because of the reluctance of most players to use the full range of dynamics on their percussion instruments.

Thus the "bad" name of drummers and the novelty of drummers who sing.