Were they really that significantly more secure? You still needed to do regular maintenance on the underlying image, etc. Same with Docker. The only big difference I see is that yes, breaking out of a container is easier than out of a VM. But are there any other significant vectors I should be aware of?
They were not more secure, just more isolated. The challenges are different.
Containers are just namespaced processes that share the same kernel as the host. A host has access to all container processes, uids, gids, file systems, and networks. Cgroups are used to limit resource access.
To run containers securely you need to understand how to protect running processes. You need to use unprivileged users where possible, drop all kernel capabilities not required, run Linux Security Modules (AppArmor, SELinux) to prevent processes from doing things they shouldn’t; and, run containers based on the smallest image possible, since a container should only have files that are absolutely required to run a process, and nothing more.
Even when you do it all right, in a multi tenant environment, it’s not safe to run all containers on the same hosts.
So if I am understanding this correctly the challenges of setting up a secure linux VM and a container are more or less the same?
The point about multi-tenancy is absolutely understandable. Isn't this an old story from the PHP world with multi-tenancy? I think a good generalization is: don't run on multi-tenant systems if you do anything (!) critical (e.g. authentication or payments)?
But that of course disregards the fact that when people _can_ do something, they _will_ do it even though they shouldn't (like running E-Commerce systems in multi-tenant environments).
Another thought regarding isolation: aren't VMs essentially just running on one host as well? Is that why you said "VMs are _more_ isolated"?
That's a very important difference, because isolation and the associated increase in overall security of the system is a core purpose of any virtualization technology. Docker promises a lot here, but a lot of those promises remain unfulfilled in reality.
Yes, containers are inherently easier to break out of than VMs, but even with that caveat there is room for improvement in container security. That alone is enough reasons for me not to be a big fan of docker in production.
But there are other vectors. With a VM you get a whole linux distribution, which of course increases the attack surface, but at the same time you also get much better isolation and that distribution's team of maintainers looking over your software, providing security patches, advisories, a simple way to update the system and so on. On the other hand there exist 'docker best practices' tutorials (not the posted one) that recommend not updating your base system at all in the name of reproducibility. Docker's solution to update management is manual image tagging and manual updates, possibly with help of external tooling. I don't think that's a good solution for that problem.
Imo the overall best solution is to run stuff in VMs and pick a lightweight distro for that.
