Terry was explicit about saying that some of the dirt lay in the word "hogeye" itself, but he declined to be explicit about what it meant.

I think we've discovered what it meant, at least by the time Terry learned it in the late 19th C. Hugill, however, seems not to have made the connection, and if the editor of the lost manuscript of bawdy shanties, if he didn't think of it, plenty of others wouldn't have thought of it either.

The phrase "hogeye man" is ambiguous anyway. It could be a man with a squint or else the captain of a hogeye barge. Or a man obsessed with "hogeyes." Singers could imagine whatever they chose.

The fact that an unrelated fiddle tune called "The Hogeye Man" is openly attested in print even before the Civil War tells me that a clean meaning (or rationalization) was not only possible but actually prevalent.

At least outside of shantying circles.

As for "navvy," the evidence is that Iam Campbell or someone like that substituted it for the other "n-word" in the early '60s.