I did hear someone argue once that Leadbelly actually borrowed an older traditional song for this. The argument was that this was a prison based chain or labour gang song. The older inmates were not effective as labourers so they had some of the less labour intensive tasks. One of those was carrying the water bucket. Being older they had grey, or slver, hair. So the generic for the old men was "Silver" or "Sylvie". And so........

I have no idea if this is right but it's interesting.