Isn't much of the song's point that almost any phrase involving "he" plus a verb plus "my" sounds bawdy in the right context, without having any specific meaning?

Few the song's metaphors seem to exist anywhere else. Some, like "churns my butter," are more obviously suggestive than others, like "cleans off the table."

"My Handy Man," of course, wass composed by Andy Razaf and Eubie Blake, and not by the singer Ethel Waters.