I have lived in/near Cambridge for 50-ish years. I find the answer is that there is not just one building called The University, but the University is a federation of all the colleges. Have never met a tourist unable to grasp that as a concept.

A variant on the flightpath one is, Why did they build Stonehenge so far from the airport?

~M~