Liz Crampton was his "mistress" I suppose. Brian and Maureen never divorced but were not "together" in that sense for a long time.

Brian didn't want to upset the family life his kids had and then in the end I think everyone just got used to Liz and he not being married so I don't believe a marriage ever happened.