I'm sure the popular belief is balls. Most roundshot was usually kept in lockers below decks (where it helped ballast the ship but was also prone to rust - hence the tedious task of chipping the rust off the shot at regular intervals to make sure it fitted the bore, as often mentioned in the Aubrey and Maturin books of Patrick O'Brian), and brought up when action was imminent.
Shot kept near the guns was generally stored in baulks of timber with indentations like a multiple egg-cup, deep enough to stop the shot rolling off if the vessel pitched, but not so deep as to make extraction impossible.
You can see such shot garlands on the wonderfully-restored HMS Warrior in Plymouth harbour.
As for the brandy - "...an' a Drop o' Nelson's blood wouldn't do us any harm..."!