When inuits and desert tribes adapted, my understanding is they were either able to adapt on their own timeline (moving gradually into harsher climates as they figured out how to stay in their summer camps longer into the fall) or they suffered immense population loss during the move, and it was a small number of survivors that rebuilt the population. There aren't a lot of records from when humans first moved to now-populated parts of the world, so I could be wrong.
That said, climate change is going to force us all into rapid adaption as plant and animal life die off, glacial streams and rivers dry up for good, sea level rises, etc. It will be easier to survive if you can afford the more expensive food and water, can afford to install the most-effective fire break around your home, can afford the expensive land in still-habitable places.
The thinking though feels a little narrow. If the non-wealthy (i.e. the vast majority of the planet) are driven to tribalism, rioting, war or otherwise die off what sustains the "wealthy"? A vast underground bomb-hardened prepper cache?
The wealthy fleeing global warming feels like it will merely be a footnote in Chapter 1 of what will eventually play out.
I worry not simply for the planet but for my children, my grandchildren, and yours, all of ours, as well.