You all know the story behind this song right?

Doc Pomus was at his wedding reception watching his lovely new Bride dancing with everyone but him. He couldn't dance with her because he had had polio as a child and walked on crutches with braces. He penned the lyrics to this song on a napkin while watching her and his family still has that napkin. He was saying to his new wife "Have a great time dear, but it will be me that takes you home' with all the implied sexuality you'd expect from a groom on his wedding night.

lovely true story. I saw his daughter explain it on a PBS special about The Brill Building years.