Nope. I was a member of Cyril's club for a couple of years, and I asked, and he said it was the more organised vendors whose competition broke the one-man-band Oggie Man.
He also pointed out that the song was a warning that nothing lasts forever, as witness the gateway ("He came through the gateway...") had also been pulled down. If memory serves, it was called the Royal Albert Gate at Devonport Dockyard. Demolished about 1970-71.