I was only ever familiar with the Eric Burden/Animals British variant, c.1963 - and assumed it came from some American bluesman :

" "Baby Let Me Take You Home", a song credited to Bert Russell (a.k.a. Bert Berns) and Wes Farrell ..... The song is an arrangement of Eric Von Schmidt's rendering of "Baby, Let Me Follow You Down" as covered by Bob Dylan, on his first, self-titled, album. The Animals' version opens with ..... The result was a key influence on Dylan's change to electric music and to the folk-rock genre....."
https://en.wikipedia.org/wiki/Baby_Let_Me_Take_You_Home


" ..... Von Schmidt [1931-2007] co-authored with Jim Rooney "Baby, Let Me Follow You Down: The Illustrated Story of the Cambridge Folk Years"..... Von Schmidt is widely (and erroneously) credited as the author of the song, "Baby, Let Me Follow You Down", which was for years a staple of Dylan's musical catalogue..... In fact, von Schmidt had adapted the song from Blind Boy Fuller and credited Reverend Gary Davis as author of "three-quarters" of the song.[5] His 1979 book about the Cambridge scene is titled after the song..... "
https://en.wikipedia.org/wiki/Eric_Von_Schmidt



Cheers, R-J