I've had some experience with Tascam porta-studios. I know that there's a lot of high tech digital stuff out there on mini-disc and dat, but for the money, you can't beat the usefullness of those little Tascam recorders. Plus, there's something about analog sound that just seems to sound warmer and better, if somewhat hissier.
There's a lot of used ones out there, and if the heads are in good shape, they can be a good deal. Just make sure that the heads aren't worn out and the sliders are fairly clean.
One thing I like about them is the easy learning curve involved with getting results out of them.
I'd suggest that you also get a vocal compressor from DBX or somebody else, to get more consistent vocal levels, and a reverb unit. That should set you up to do quite a lot, even a releasable demo recording.
Another way to go, is to see if you can dig up a four track, or better yet, and eight track reel to reel. I know Teac/Tascam made them, and if you can get a good deal on a used one, the larger tape format will give you more "headroom" and less tape "bleed". You can also fit a heck of a lot of sound on one tape reel.
People are tending to hang onto them these days because analog sound is making a comeback, and the recorders had lots of features. Still, they're out there somewhere. Good luck.
Make them tunes, dude.