My mother remembers the song as having been recorded by Alan Jenkins. As she remembers it, the verses and the end were spoken and the chorus was sung. She remembers the lyrics as being:

Chorus:
There's a man that comes to our house every single day;
Papa comes home, and the man goes away.
Papa does the work and Mama gets the pay,
And the man comes around when Papa goes away.

The man that comes to our house, he's very nice.
He comes in the morning to bring Mama ice.
But the teeny-weent little piece soon melts away,
so he has to come back again later in the day.

(Chorus)

The man that comes to our house, he comes to bring the milk.
He walks right in the kitchen and talks as smooth as silk.
I always have to hold his horsey down by the gate
'cause he always stays so awful long that horse don't want to wait.

(Chorus)

The man that comes to our house, he comes to get the trash,
in a little white jacket and a little black moustache.
Now, it's all very strange, but it always seems to me
He's a little more familiar than the trash man ought to be.

(Chorus)

When I grow up, I never want to be
A carpenter or doctor, no sirree.
Don't want to be a lawyer with an office downtown.
I just want to be the man that comes around.

(Possibly another chorus)

I hope that helps.