There used to be a sort of poetical cliché of a working man being a "horny-handed son of toil", or some such. I believe Lord Salisbury famously used the phrase in Parliament during a C19 House of Lords debate on working conditions. So it surely implies a skin hardened by hard work or labouring in harsh climate.

≈M≈