A 'man' is a man-o-war. If it's mishandled with a lot of sail aloft, or a real sudden shift of weather, the wind can lay it over on its 'beam ends' ie. on its side. Thus 'blow the man down'. Been there been thru that, tho not in a man-o-war.

If you ever go to Stockholm, you might want to see the Vasa Museum. This warship sank from exactly this less than a mile into its maiden voyage. I've created a song about it, here's the tune lyrics to be published (soon...)