Email from Ken Whiteley who has been singing this song for 35 years says that German is a town near Memphis that probably has/had a prison.

The prison connection makes sense if you listen to the words.

A quick check on Mapquest shows a place called Germantown close to Memphis.