Best option these days is to get an MP3 player. Most of them do recording as well, either from a built-in mic or from a line-in connector, but check before you buy. AFAIK iPods don't, but most others do (especially the smaller Flash ones).

I've just got an iRiver T30. The built-in mic is a little muffled, but it's not bad, and line-in just records whatever you give it. The best part about it though is that the recording (and playback) is utterly noise-free - no static at all! I thoroughly recommend it.

Graham.