Steve Shaw is correct about the music that Brahms wrote late in life. Brahms had a great gift, a tormented personality, and was a self-hating perfectionist. He wrote string quartets, but we will never know what they were like because he burned them. It seems, judging from the intense and imperfect music of his younger years, that Brahms just had a lot of crap to get out of his system.
One of the areas where Brahms does good work, and not known enough, is music for a-cappella voices, or choruses with minimal accompaniment. Here, Brahms immersed himself in Bach scholarship and looked closely at the polyphony and counterpoint of Bach's choral works.
Lieder is another place to look. Yes, Brahms had stuff to get out of his system as far as voice-and-piano music; his early songs have some imitation Mendelssohn in them. It is interesting, though, to watch Brahms mature as he goes on composing Lieder for solo voice. Something about being truthful to the poetry and the lyrics that he sets, causes Brahms to curb that tendency to wallow and go to excess that spoils some of his instrumental pieces.
By the time Brahms is old, and gradually ailing (he had a lingering dying process, as his cancer was misdiagnosed at first), he is writing solo piano pieces, and art-songs for voice and piano, that are almost twentieth-century in their incisiveness and brevity. There are fewer places to hide in this music, for the performer, the excess has been trimmed away and there are no "vamp-'til-ready" fillers. The late Intermezzi take my breath away, for example.