Agreed, I always thought VPC and all that complexity was a big step backwards. My org is moving from a largely managed network into AWS, and now we have to configure the whole network and external gateways ourselves? What engineer wants to do this?
VPCs are virtual, but I don't need VPCs, I need the entire network layer virtualized and abstracted. As you suggested,just grouping devices in a single network and saying "let them all talk to each other, let this one talk to that one over this port/IP" should be all I describe. Let AWS figure out CIDR, routing, gateways, etc.
People use it as a (imo lazy) form of enforcing access control. If two services aren’t in the same VPC, they can’t talk to each other. It theoretically limits the damage of a rogue node.
Of course, it also creates a ton of overhead and complexity, because you still have to wire all your VPCs together to implement things like monitoring and log aggregation, for example.
As other people have suggested, the better solution (imo) is to have all your traffic be encrypted with mTLS, and enforce your ACLs with certs instead of network accessibility.
However, if you are relying on defense in depth for security, then having them be network separate helps prevent internal DDoS attacks, whether malicious or not.
Enforcing security across the entire network layer has many positives. But I don't want to be messing with the lower levels, and those lower levels all have the same security models and solutions as one another, at least if you view them at a high level.
VPCs have value as a security and availability solution, I just don't want to have to configure it to get what could be an automatic benefit.
VPCs are virtual, but I don't need VPCs, I need the entire network layer virtualized and abstracted. As you suggested,just grouping devices in a single network and saying "let them all talk to each other, let this one talk to that one over this port/IP" should be all I describe. Let AWS figure out CIDR, routing, gateways, etc.