The Zoom H1 is a new successor to the H2, only recently available. Though I haven't tried one, I'd guess it's perfect for this.

I know people who use cheaper voice recorders with success - personally I'd want to check the recording quality carefully before purchasing one because some of them really are voice only, and sometimes a reasonably clear recording is needed to discriminate the tune from the background noise in a session.