I think instead of "had me a man or two" is "had me John the Conkeroo" - typical blues reference, John the Conkeroo (also Johnny Conkeroo) is an adaptation of John The Conqueror and refers to a small statue used as a good luck charm.

The lyrics you are referencing are from the song "When I Get The Time" I have it on an old Siegel-Schwall album on Wooden Nickel Records. You can get the three disk set, from Amazon, that has it on it now.