just came across this:-

http://www.southafrica.info/travel/advice/saenglish.htm

"skinner (skinner) - Gossip, from Afrikaans. A person who gossips is known as a skinnerbek"

So (if there's a connection) maybe a muleskinner spent a lot of time talking to his beasts? Or about them?