Humans aren't rabbits. Humans tend to have children in inverse proportion to the amount of grass, poorer people having more children than richer ones. Of course it's not a direct correlation, or a simple one, but the effect is there.
More houses in locations people want to live in, yes. In the end, even leaving house prices aside, a larger population needs more housing, and the UK has been underbuilding several tens of thousands of units a year for decades.