I'm not a hardware person at all, but my understanding is that distributed computing systems are hard to scale due to latency, and a huge factor in latency at this scale is physical distance.

This was also mentioned in the AI Day presentation, and listed as a major factor in the design of Dojo.