Sounds like the "squelch" on the receiver is not set correctly. When the transmitter is turned off (or the batteries die) it should mute.

So turn the transmitter off, and turn the squelch control to the point where the background noise just cuts through. Then turn it back a bit.