Would love to see a more detailed write-up, if one exists, on your experience. We're nowhere near your scale but will probably ramp up in the next year or so.
If you ever open-source your rightsizing scripts, I think a lot of people (including myself) would find that useful. While the vertical pod autoscaler attempts to solve this problem, it's incompatible with horizontal pod autoscalers based on CPU and memory, and thus unusable in a lot of cases. Going with a semi-automated solution might be ideal.
Very nice! I suspect the economy of Cloud will soon change in such a way that Cloud providers will be focusing more on Enterprise, which have very little capability of doing similar things. And then, a lot of companies like Hakuna Cloud (reported here) will emerge, just like what happened in the last iteration of IT infrastructure.
The number of running nodes is a direct function of the number and size of your containers. The cluster autoscaler stops and and start servers based on how many containers are desired, which is in turn based on the Horizontal Pod Autoscaler inputs (e.g. incoming requests).
Yes, we own some datacenters as well as use multiple cloud providers. You are still responsible for running your own worker VMs with a managed Kubernetes anyway.
as in hidden costs- costs that the organization is unaware of because it's wrapped up in r&d/goods delivered on the accounting sheets. Its='s not until you pull in the utilization metrics that it reveals itself as waste.
- Using the Kubernetes Vertical Pod Autoscaler (https://github.com/kubernetes/autoscaler/tree/master/vertica...) for CPU and memory scaling, and switching to metrics like requests per second and connection count for horizontal scaling
- Collecting metrics on container CPU and memory allocation vs utilization
- Writing scripts to auto-submit PRs to SREs with better recommended sizing based on actual usage
- Tuning our VM instance sizes and autoscaler configs
A few engineers were able to save the company several times their salary with a few months of work, and plan to 10x the savings over the next year