Drums are loud because they are efficient devices for converting hand and arm and foot movements into vibrating air particles.
The people who play loudly and show no awareness of other musicians and audiences are not limited to drums and percussion instruments.
The problem simply gets back to the inherent efficiency of drums and percussion. 20 drunken violinists are as bad to listen to as 20 drunken djembe players.
The other problem with drums is the learning curve. The curve for drums starts very shallow, almost horizontal. This means that initially, everyone can "beat" a drum. The curve, however, rises steeply as one proceeds along the path from "beating and hitting" to playing the drums. Most people dropout as musicianship levels need to increase and daily practice and hard work become necessary. A few continue on and on because excellent drumming is its own reward. These people however are lumped in with the "beaters" who never move away from that level, rarely play musically, and who give all drummers a bad name.
The uniquely shallow initial learning curve for drums attracts the drongos. And then they play loudly.
This is unfortunate for those people, such as myself, who see and hear as much meaning and passion in a well played drum as there is a well played violin, flute, whistle, voice, cello, etc.
So, it is not the drums that drums are noisy and annoying. In fact, they are very silent. It takes the people, their hands, sticks, and feet to bring them to life. Whether the sounds are good bad or indifferent depends on the people.
The issue of whether or not to have drums reflects more on the nature of humanity than it does on membranophones and idiophones. The conflict in the middle east (or anywhere for that matter) has more in common with this argument than any argument about melody and harmony dynamics, pitch etc.
People kill music drums do not.