Not to mention the fact that by the third century, if a general controlled enough troops, they would (and did) overthrow the Emperor, so the Emperor tended to run around with one large army putting out fires, which was inefficient.
And many of those soldiers where mercenaries from the outside, who, by serving the empire, where put in the position to loot the empire's superior agriculture.
This went on for many generations, it's not a "last days before the ostrogoths" exception. Competing "emperor startups" in an empire representing pretty much the entire known civilized world, surrounded by mostly harmless outlaw wilderness. It must have felt as natural and "could the world even be not like this?" to the people populating that world as nation states, competing militarily, economically and in the Olympics seem natural to us.
