As soon as there are enough houses, the cheapest of them will become "affordable".
So I say build new houses, but only at the top end of the market. That way we build nicer houses, improving the housing stock, improving living standards and making the country a nicer place.
Then everyone moves up a notch, leaving the cheapest houses for first time buyers.
Surely that's much better than building nasty little boxes at the bottom end of the market?
Oh, and a few less people coming in would help too.