It's Boxing Glove. I've known the song for half a century, since my mom read me "Davy and the Goblin", which is where it first appears. Davy is a sort of Alice In Wonderland story, where a boy goes on a "Believing voyage" and meets characters from fairy tales, including Little Red Riding Hood and her father, Robin Hood. When the goblin introduces him to Sindbad the Sailor, Sindbad sings "The Walloping Window Blind", one of several nonsense songs in the book. You will note that the party is off on the morning train, to cross the raging maine. Crossing the sea in a train would be more than a bit damp, and he is indeed, off to his love with a boxing glove then thousand miles away.
Julie Andrews sang it on the Muppet Show and punched a muppet each time she sang the chorus.