I remember trying to work out scenarios in which the mitochondrial single female ancestor did not have to be the only human female alive at the time - she only needed to have female offspring slightly more fertile than others at the time in order for all modern females to have descended from her. Or slightly more attractive to mates. Or they needed to have more girl babies than boys, while others had more sons. The same applies to the ancestral Y-chromosome male, who does not even have to have been alive at the same time and in the same place as the mitochondrial ancestor.

It gets a bit boring randomising generations by tossing coins and dice after a while, so I didn't carry on with the exercise, but it was quite clear that the offspring of any one individual could come to dominate the group. Anyone can try it, recording each generation.

1. Choose a number of females to be in the group.
2. For each female, toss a die to determine the number of offspring surviving to breed, replacing six with 0. (If you think that 5 isn't enough breeding offspring, use two dice, but I would think that infant and maternal mortality would tend to keep numbers low in a primitive society.)
3. For each child, toss a coin to determine gender.
4. Repeat the exercise from 2 with the next generation.

You could do the same for males, but you'd need a much bigger difference in offspring numbers if you think that one male might be running a harem. A lot of zeroes, and one male with a huge number of offspring would reduce the number of generations to get down to one male ancestor of all males dramatically.

Penny