I'm a geologist, and I like the way "the Society upon the Stanislaus" reflects nineteenth century life. In those days the world was agog at the discoveries of fossils, especially dinosaur fossils. Newspapers on both sides of the pond reported on discoveries in Kansas and Colorado, and no doubt exhibits and lectures were thronged.
The "old red sandstone" in the poem is a joke about the Old Red Sandstone, a famous rock formation. The Encyclopedia Britannica says, "Old Red Sandstone, thick sequence of Devonian rocks (formed from 416 million to 359.2 million years ago) that are continental rather than marine in origin and occur in northwestern Europe, Scandinavia, Greenland, and northeastern Canada."
There's more interesting information in the Britannica article. If you are ever heading for Denver, Colorado, see if you can visit the Sternberg Museum in Hayes, Kansas, where you will see some of the famous fossils that set the world on fire back in the day.