I grew up in Maine, where "peg" was a verb, meaning to throw, as in "I pegged an apple at his head." Since so many Mainers went to sea, perhaps this got used afloat, with the result that "off the peg" meant to miss your target.
Thinking outside the compass,
Jerry