Grab, I keep a little dictaphone handy for just those moments you describe (this experience isn't unique to you). The beauty of it is that it requires no setup -- it's sitting on a table in my living room, and I just turn it on and point it in the general direction of my guitar and voice when I want to capture somthing. Then I can forget about it and move on, confident that my flash of inspiration will be there when I want to come back to it.
As for sound quality, it sucks -- but I'm not doing it for sound quality, just for a memory aid. For that purpose, it works fine.