I'm currently doing two different stage shows in one week, so it's not just having to remember all the words and notes, but to enter stage at the right time, collect the right props on the way in, do the right moves in the right places, keep on singing while you're manipulating a maypole, dancing, sing lying flat on your stomach, get up, sit down, kneel, climb steep stairs, run down ramps, or whatever else the director wants.(And keep on singing and smiling!)
Learning it all is repetition, repetition, repetition: the music and words by having the CDs going in the car, and singing along. My poor brain is addled, but it is amazing that the brain can deal with so many things at once!
Don't stop singing Drumshanty: you have a lovely voice: use the words if you need to, especially if it's "just" a session or singaround.
Having said that there are plenty of performers out there who do have the words in front of them as a sort of comfort zone (as opposed to those who have their noses right into the book or file and are reading them as for the first time): they may not need to use them, so can still deliver the song well, and still make contact with their audience. But this takes off the pressure and worry of possibly forgetting (which in itself makes forgetting more likely!), when a quick glance at the words will get them back on track. I personally don't find this done subtly a sign of inferior performance.