> easier to maintain at the container level than recreating images and redeploying VMs

I'm not sure I understand how that could be, since the best practice alluded to elsewhere in the thread is to recreate and redeploy (as part of ones normal CI/CD process, assuming that even exists) containers as a way of keeping up on security updates. How is that different, let alone easier, than doing so with VMs?

> startups just want to run apps and aren't doing any sophisticated lower-level infrastructure

That may be important at the very earliest stages, but that kind of concerted ignorance can be dangerous.

> it also replaces several other accessory software

The "also" being in addition to abstracting away all that tedious, "accessory" software. The problem is that, abstractions are leaky. At some point, it may well become important for someone in that startup to understand how/why the accessory works because it suddenly became critical to the business.

Of course, that point may never come, if the startup doesn't survive that long, so why bother thinking that far ahead?

Kubernetes is a solution for 90% of the effort and services needed for distributed apps running and interacting with each other over a cluster of servers. Those accessory services aren't abstracted, I never used that word. They are instead provided out of the box by K8S, leading to fewer individual components to run, maintain and monitor yourself. You can switch out components at any level, from a single container running your favorite webserver, to a full service mesh, to your own custom controllers.

Containers are much smaller than VMs, all the way down to just a single binary for your application if you want. Definition files are smaller. Restarting or replacing a container is much faster.

Your comment amounts to a big "what if", but this isn't a complex topic. Focusing on what you actually need is what leads to success. Any competent technical leadership will plan ahead and care about low-level detail when necessary. But you definitely don't need to unnecessarily worry about it, and that's what Kubernetes helps with.

Of course you can skip K8S altogether and have your setup, but that is rarely needed because it can now be solved with a common industry framework instead of bespoke solutions or special PaaS provider APIs. None of this is revolutionary, and no different than any other cost/benefit analysis for build/buy of any other component.

> They are instead provided out of the box by K8S, leading to fewer individual components to run, maintain and monitor yourself. You can switch out components at any level, from a single container running your favorite webserver, to a full service mesh, to your own custom controllers.

Fair enough, though I was misled by the word "replaces", which, to my mind, means something different than merely providing out of the box.

> Containers are much smaller than VMs

I don't disput that. I'm also well aware that size matters. However, you initial assertion was that it "removes almost all ops overhead" and that it is "much faster and easier to maintain" (emphasis mine), which merely increasing performance (as performed by computers, not humans), no matter the degree, doesn't do.

I'm still not seeing how the maintenance (done by humans) is any easier (or, for that matter, faster) than with VMs.

> Your comment amounts to a big "what if"

Only on the last sentence asks such a question. The rest of the comment asks different questions, some of which you've addressed above.

> Any competent technical leadership

This is a bit too "true Scotsman" to be useful. The question is what actually happens with actual leadership once a particular tool is in place. Does it encourage (inadvertently or otherwise) long-term dependence on the tool's ecosystem to the exclusion of those bespoke changes, or does it naturally peel away the leaky abstractions once they no longer hold value?

There are lessons to be learned from ORMs. Even early on, there was little (no?) controversy that the best practice would be to start off using them everywhere initially and replace them as needed. I don't believe this practice was followed, despite the pain, at least partially due to the perceptions of ease and consistency in sticking with ORM-everywhere.

> But you definitely don't need to unnecessarily worry about it

You may be conflating premature optimization (a strawman you detail in the next paragraph, which I won't address) and merely considerting the future, the "what if we succeed?" question, which is what I'm advocating.

I say it's quite necessary to at least think about the consequences of a tool choice in the context of eventual success, especially if the initial investment is high, but even if not.

All that said, I hope you've noticed, I'm not actually making any assertions about Kubernetes, just asking questions (and challenging assertions). I tend to take a position of skepticism with anything that seems to gain popularity through network effect, as it becomes extremely difficult to separate facts from hype, even in (especially in?) anecdotes.

"removes ops overhead" is about K8S, not containers. Less stuff in a container on a bare VM means less to manage and maintain, thus easier.

I don't get your point in "considering the future" - what are you saying exactly? Yes people should plan for it. You have to be competent enough to do that, and if you're not then that's a different discussion.

However, even if you have a poor team, then Kubernetes still helps because it's well designed, flexible, reliable, and can grow with your business. As stated, it's incredibly customizable. The entire system is designed around interfaces, like the CRI for container runtimes, CNI for networking, CSI for storage, and more. In fact the extensibility APIs like metacontrollers and CRDs are so powerful now that some of the main primitives (like StatefulSets) can even be recreated in just a few lines of code.

Given that, your question of "what if we succeed" (which must be tempered by the fact that very few need high scale, and fewer still will outgrow the capabilities of K8S) is answered by "you upgrade the things you need". It really is that simple. Kubernetes is well-designed, documented, battle-tested, and supported by a large community and major vendors, so it's a better choice for both early and late stage deployments.

Considering the major investments by every cloud vendor and the benefits detailed by hundreds of companies both large and small, I believe this has proven itself beyond just a hype cycle.