Thanks for these lyrics I've been trying to hunt them down..
As for the stamps it's just an old fashioned term for the money you pay in national insurance contributions. When it was first introduced in 1911, people would go to the post office and literally buy a stamp by way of paying their contributions, they would then show those 'stamps' as proof when claiming any benefits.