I use a very tightly rolled piece of sponge pushed as close to the neck as possible behind the dowel stick. If I leave it out, I get some hollowness in the tone, as if I'm playing it in a huge concrete room. When it's in, the banjo sounds no quieter, but beautifully mellow.

Cheers
Ray