The population of a place doesn't have to double, or even change at all, for prices to go up. All that is necessary is for more people to want to live in a place. Those with the $ to satisfy that desire will outbid those who don't and you can easily double/triple/10x prices with no change in population.
Your factual statements are wrong. The US population isn't flat over time, and housing (supply) decays over time. Prices aren't up everywhere, just the places people actually want to live.
Your figure doesn't contradict my claim. We have spare housing capacity in undesirable places people don't want to live (rural towns, small midwest cities). We have shortages in the (especially coastal) cities, where the jobs are. This is obvious from prices, which are set by supply and demand.
