I always understood that "cold enough to freeze the balls off a brass monkey" was originally a naval expression. A brass monkey was the rack on which canonballs were stored, and the original saying was "cold enough to freeze the balls ONTO a brass monkey".