Thank you to John Moulden for his quote "It is attested that ships in the Quebec or Maine timber trade sailed to Ireland and Britain with timber, .... rather than sail back in ballast for more wood." It fully explains the source of the error.
The term "In Ballast" was used in the 19th. century to describe a ship sailing without a cargo. If returning with passengers the ship would still have carried ballast, but would not be described as "in ballast" because it now had a cargo.

All the best

Jim Hancock