there were more birds in one look than the whole other hour that I counted. I don't see how it works.

Two ways, JHW. The dull and boring one is that the birds exist. So if they were in your garden the next morning they were somewhere else the day before - as like as not in someone else's garden if the times of day were the same. Add enough samples together and the unusually high and low sets tend to cancel out.

More interesting is that counting the birds is only one of the aims. It is at least as important to encourage people who would not otherwise do so to take an interest in the wildlife just beyond the door, which over the year can have a beneficial effect on all kinds of creatures, not just birds.