Not Cornish, Joy: it's about coal mining. I've always taken the verses to refer to Durham, Lancashire and South Wales respectively

"Pitmen" are from the North East of England, where they speak "pitmatic" (and the word sark is not unknown: Dolly Coxon pawned hers).

When I cam to Walker wark,
I had ne coat nor ne pit sark
But now aw've getten twe or three,
Walker pit's deun weel for me.

"Collier" seems to have been the preferred designation of Lancashire miners (especially in song).

Doubtless, MacColl could have written other verses to include Scotland, Yorskhire, etc., but felt that three were enough.