I am reading Iron Kingdom, a history of Prussia, and was struck by how often it gained and lost territories. After every war (and Prussia fought a lot of wars), it won/lost chunks while also gaining/giving up chunks elsewhere. Silesia, Hanover, Saxony ... the various states that it absorbed/traded lands with are difficult to keep track of. Heck, "Prussia" was originally the term for land east of the original land, centered around Berlin, that the Hohenzollern monarchy ruled; after annexing it the term for that bit of new land somehow became applied to the country as a whole.

The Silesians, Hanoverians, Saxons, etc. all viewed themselves as different people from the Brandenburgs/Prussians and each other. As toyg said, ethnic-based nationalism—viewing themselves as "Germans" and thus deserving a single German nation—is a 19th-century phenomenon; further, the various German states (not excluding Prussia) viewed and suppressed such nationalism as subversive. Mutual cooperation nonetheless grew during the century, notably the customs union.

It's something that really started mid-XIX century, with the emergence of the concept of ethnic-based statehood in France and Prussia. The process continued unabated for about 100 years, until the populations were divided "cleanly enough" in most of the continent. The end of WW2 was just the last chance for big moves in that direction, and some countries grabbed it with both hands.