It was not unusual in early times to consider a marriage not fully consummated until a child was born.
The offer to "wear hat and glove" until the child was born to the wife, indicates that the marriage was considered to be only conditional up to that point, thus to have an official lover would not count as adultery.
Note that this is the man's point of view, the woman does not quite see it that way!
Cheers
Dave