The earliest dated mention of Cotton-eyed Joe I have found is in Perrow, a song from the MS of Dr. Harrington, 1909, collected from Mississippi Negroes.

Ef it hadn't been fer dat Cottoneye Joe,
Mought er been married six er seben year ago.

E. C. Perrow, "Songs and Rhymes from the South, 1915, part VIII, no. 81, Jour. American Folklore, vol. 28.

Scarborough's notes suggest that it could go back to slavery times. It does seem to be a Black song.

Common slang usage, as stated here or in another thread, defines cotton-eyed as having the whitea of the eye prominent (J. E. Lighter, Historical Dictionary of American Slang, vol. 1).
Nothing suggests a white man is involved, and I see nothing to support racist interpretations.