The OED is the source for the suggestion that the term didn't come from the prison. I had in my mind that the term Clink was applied to the prison in the 16th c., but this site says the name has been found in use in 1473 (no source given). That is earlier than the OED quote. I dunno!
Southwerk
www.aaron.atte.southwerk.mcmail.com/AaronFrames/hyst7.htm