I've possibly found the earliest mention of "Cotton-eyed Joe" yet.

8 May, 1875 _Saturday Evening Post_

A fictional piece, wherin the young white heroine is singing this song while cooking with her Black nursemaid/cook. She says that the Black nursemaid taught her the song. The nursemaid says "hush. Don't sing that" knowing that the girl's mother wouldn't approve.

The line she sings is "Don't you remember a long time ago, I dreamed that I ran away w/ Cotton-eyed Joe?"

Later in the story she sings "Oh, I'd have been married twelve months ago, if it had not have been for Cotton-eyed Joe."

Next, who comes to the door but her blue-eyed cousin, Joe.

Later in the story, a character describes Joe as a person with "great white eyes."

But, still later in the story, Joe is described again as having BLUE eyes.

So, perhaps the song was originally put to words by African Americans, obviously prior to 1875. But whites certainly knew the words by 1875. I can't see any derogatory racial meanings here. The term could have reference to both persons with prominent whites of the eye, and also could refer to blue-eyed persons. Or both.