Finally got the words from my mother. Apparently she thinks she has all the words, but wants to know the source of the song.

DARLING CHLOE

In the old Carolina state,
Where the sweet magnolias bloom
And the pickaninny darkies are all gay,
There is one I long to see
She was always good to me
And I left her many, many years ago.

(second tune)
In the cotton and the corn
Where we both were bred and born
And together in the fields we used to hoe,
It's been twenty years or more
since I crossed the cabin door
And I'm going back to see my Darling Chloe.

(refrain)
Darling Chloe, Darling Chloe
Your sweet face I soon shall see I know.
Where the sunny southern breeze
Fans the tall palmetto trees
I am going back to see my Darling Chloe.