ARM Linux isn't really slimming down, so the chips need to grow to catch up.

You're starting to see devices like the Renesas RZ/A1H that has 10MiB of SDRAM on the die and can boot Linux out of a single cheap QSPI memory part.