They found remnants of a "mouth bow" among the Neanderthals. It has to be one of the oldest at least.

Unfortunately nobody knows the music they made.

What about percussion instruments? Did ape predecessors beat out rhythms with twigs and dance around shrieking tunes?