"yeller girls"...refers to obvious 'mixed race' situation...you'll see the phrase in many songs and references...There was a time when 'real' black men were held in contempt for associating with them, though there has always been a high degree of ambivilence....

("High Yeller" can refer to any person of a certain degree of mixed race, and their place in black society is quite complex)