I think that’s true, but also it’s not clear what tools a startup should use. Even if they stick to a single VM, managing that VM in a way that is reasonably safe and reproducible is somewhat important—the more often you have people manually SSHing onto the VM the more likely you will need to recover functionality from scratch while also making it almost impossible to do so in any timely manner (because the functioning state of the VM is the result of many thousands of manual changes, many of which are undocumented, incompletely documented, or incorrectly documented). So then you can start learning about automating changes on a machine, but now you’re into territory that is about as complex as Kubernetes (or perhaps not “complex” as much as “you have to figure out what combination of tools and practices will work”, while a cloud-provider’s managed Kubernetes makes a lot of these kinds of choices for you out of the box and many others are obvious).
Similarly, you might reasonably try to use a PaaS, and it might work at first, but then you need to do something as common as a background task and find that your PaaS doesn’t support things like that (you can try running it in a process in your PaaS’s container and just accept that the background tasks are likely to be interrupted periodically?).
I don’t think Kubernetes is the
answer for a startup, but it’s not obvious what “the answer” is, and much of the discussion ignores the complexity in the alternatives. It’s not helpful to say “Kubernetes is too complex” without articulating a simpler alternative (unless you are hinting at a market opportunity).
I think it's clear to people who specialize in that. I don't know why startups don't hire a DevOps/SRE/Syseng/Sysadmin/Architect contractor right off the bat. Hire a really experienced one for one month and they'll give you several different options, lay out the pros and cons, costs, staffing, time estimates. Take their suggestions and implement the one that fits your business plans.
For example, a single VM you ssh into works fine if you do daily VM snapshots and deploy from snapshot. If somebody breaks it, revert to last working snapshot. Blue/green in multiple environments work with it too. Super cheap, super simple, easy to recover, automated. Very boring.
> I don't know why startups don't hire a DevOps/SRE/Syseng/Sysadmin/Architect contractor right off the bat.
It's entirely likely that they are doing this, but the DevOps/SREs that they're hiring are prescribing the tools that they know and have seen work in the organizations that they're coming from.
I've been an SRE for nearly a decade, and I honestly couldn't tell you in detail how to set up a pet-VM deployment with automation and guidance about who deploys to it, on what cadence, etc or who operates it, etc. In my first DevOps job, we were doing something like this, but it was a hot mess--that startup hired the wrong sysadmin to bootstrap us, and the next guy they hired moved us to AWS ECS (running on Fargate) and that was a dramatically more effective solution for us. The point isn't that ECS > pet VMs, but only that we were able to find a successful path with ECS where we were unable to with pet VMs despite hiring someone who allegedly had that experience. For that organization, ECS (rather than pet VMs) was the Very Boring Solution (mind you, I'm not a particularly big fan of ECS either).
> I would downvote you if I could because what you wrote is just ignorant and anyone who dealt with k8s before can tell that you never dealt with it.
I don't know why you're snarking so hard when there are dozens of managed Kubernetes offerings out there.
> It requires a lot of hardware just for the base, 9 servers minimum.
An HA deployment requires 3 nodes for the control plane, and as many or few workers as necessary. But yes, this is 3 more nodes than you would run without automated orchestration. Of course, the cost of these 3 nodes is amortized over all of the worker nodes, so if you're only running 3 worker nodes you're paying 100% for orchestration, but if you're running 30 worker nodes you're only paying 10% (this is assuming equally sized nodes, for simplicity).
> The more complex something gets the harder it is to get done right.
Right, but if someone else packages that complexity up for you in a piece of software (like Kubernetes or Linux) it's often much easier to get it done right than it would be to roll your own from primitives.
If 9 nodes is a lot for you, then I agree that your problems are small enough that manual solutions can be much simpler than using Kubernetes.
You also don't need an angle grinder to open a cardboard box, but that doesn't mean angle grinders are stupid and can all be replaced with a box cutter.
You’ll need to name names and their particular shortcomings if you’re going to say there are no good PaaS alternatives. In particular, both AWS and GCP have cron-as-a-service, which can then be hooked up to AWS Lambda/GCP Functions. Now, I’m not saying this will work for every use case out there (eg there are time limits on both), but without getting into the weeds a little bit on why those PaaS won’t work (for your use case), there are viable alternatives to K8s’ complexity.
> You’ll need to name names and their particular shortcomings if you’re going to say there are no good PaaS alternatives.
I'm not saying there are no good PaaS alternatives, I'm saying that it's not easy to find the right PaaS alternative. You might think you have the right one, and then some key feature is just missing (in my example, the ability to kick off orchestrator-aware background tasks).
> In particular, both AWS and GCP have cron-as-a-service, which can then be hooked up to AWS Lambda/GCP Functions.
Right, but now you're dealing with cloud-provider primitives which doesn't seem significantly more straightforward than dealing with Kubernetes (instead of knowing Kubernetes, you need to know the cloud provider APIs).
> there are viable alternatives to K8s’ complexity.
I'm sure there are, but figuring out which PaaS is appropriate for your use case (or a combination of PaaS + various cloud provider tools + etc) is a hard problem, possibly just as hard as building on a managed Kubernetes offering.
Personally, I'm hoping someone makes an open-source, push-button Kubernetes distro that has the common stuff built-in: external-dns, cert-manager, ingress-controllers, network storage backends, central logging, prometheus/grafana, etc. This would give you even more of the happy path built in without necessarily coupling you to a cloud provider while also giving you plenty of "future-proofing" (if your pass doesn't have some feature, you often need to switch to another platform, but if your Kubernetes distro doesn't have some feature, you can just build it atop Kubernetes).
Similarly, you might reasonably try to use a PaaS, and it might work at first, but then you need to do something as common as a background task and find that your PaaS doesn’t support things like that (you can try running it in a process in your PaaS’s container and just accept that the background tasks are likely to be interrupted periodically?).
I don’t think Kubernetes is the answer for a startup, but it’s not obvious what “the answer” is, and much of the discussion ignores the complexity in the alternatives. It’s not helpful to say “Kubernetes is too complex” without articulating a simpler alternative (unless you are hinting at a market opportunity).