This is pure uninformed guesswork, but might the song be descended from a shanty? I've never heard it, but on reading the words one of those 'Sally Brown' sort of tunes seemed to be struggling for utterance.

Could 'Shadow day is a ripe banana' once have been 'Sally Brown's a bright mulatta?'

'Four white horses' appear in versions of 'Jordan', which is sometimes a spiritual and sometimes closer to a shanty.

Valmai