Imagine the support infrastructure farmers and family need (schools, medical, stores, some entertainment maybe). And then all the infrastructure those workers need.

It’s not like every farmer is happy living in a tent alone, traveling offroad only to deliver their crops and buy seeds in the city. They like some form of community.

It may be “inefficient” but if you want to be efficient, everyone will starve.