I tried various cheap voice recorders - essentially digital dictation machines, I was very dissatisfied with the quality of the recordings. I found the recordings had much too much noise on them so I splashed out on an Edirol R09. It was expensive but the quality of the recordings from the built in mike was excellent. It records both WAV and MP3 (of various qualities) and takes SD memory cards so spare memory cards are readily available at reasonable cost.

The R09 was what was available at the time but I see some other better quality recorders have appeared on the market which seem to be a bit cheaper so there is more choice, but on the whole, with gadgets like this you get what you pay for, IMHO.