MIT was only a few decades old at the time -- having been founded about 75 years prior. Columbia was over twice as old at the time (185 years) and was well-established as a global center of physics research.
It seems to me that the prestige of a university is largely a function of it's age. It takes time to become prestigious in the eyes of the public through word of mouth. If you look now at the research universities that are only about 70 years old or less, they are not taken very seriously and are "backup schools" regardless of the quality of research or instruction going on there. Moreover, some of the prestigious "ivy" schools have fallen off their game and aren't really top tier in any academic fields anymore, yet are still highly regarded by the general public.
