I think you are wrong. Elem St was the black people's red light district in Houston Texas. The honky tonks and music halls were located there. The song originates in the Black honky tonks, but it was picked up by a White string band, which made it popular.

The Grateful Dead used to do a slow, blusey version of it, and Garcia also did a faster country version, on one of his country albums, with Grisman. I have a tape of the original White country band record and a newspaper article on the song's history somewhere. If I find them, I'll post some more info.

"If you go to Deep Elem, put your money in your shoes
Cause those Deep Elem Women will give a man the Blues
Oh, sweet Mama! Your Daddy's got those Deep Elem Blues.

EZR