I seem to recall reading about a flood somewhere in the south in the early 20th century when African-Americans were rounded up at gunpoint and forced to work like slaves to build up the levee. I can't recall the time or place.

Does this ring a bell with anyone?

It could account for the line "They were workin' the women as hard as the men" (or something similar) that occurs in some versions of this song.

The "obvious" time and place would be on the Brazos River in 1910, but I can't find any confirmation that such a flood occurred.