Basing this on nothing but common sense, to me what the chantyman and the fifer/fiddler might have had in common is a desire to avoid the very different excesses of life aboard a greatly undermanned merchant vessel or a highly strict-disciplined naval vessel; hence, with a bit of imagination and craft, learning to lead the chanty or play an instrument, neither being necessarily highly-skilled as someone has already pointed out.