I'm with "down" as in "down wind" because that makes perfect sense, and most mariners wouldn't read charts and have any preconception about up and down in terms of north and south.

Even today, if you ask someone to draw you a map of how to get to the pub, shops etc., they don't make a special point of putting north at the top.

The version I've always "heard" is:

It's three score and ten boys and men were lost from Grimsby town
From Yarmouth down to Scarborough many hundreds more were drowned
Their herring boats, their trawlers, their fishing smacks as well
They **learned to fight** that bitter night
And battled with the swell."