Sorry, my post was rather lacking context wasn't it. What I meant was that the vast majority of my work is now done end to end on ARM CPUs and even the bits that haven't been, like GitHub runners, are starting to make ARM an option. Customers have been asking Digital Ocean for ARM servers for years [0], and every time they do, DO just tell you to upvote the suggestion on their feedback site, but you can't, because they closed it [1].
edit for additional context: An ARM server on Hetzner costs roughly 1/10th that of an x86 one on DO with the same core count and memory, and in my tests out-performs it too.
We waited on this issue for 2 years before migrating off EKS Fargate to Managed Node Groups due to this very reason. In the end, it turned out to be much better of an environment anyway, because you didn't have to deal with Kubernetes oddities (like lack of Daemonsets) requiring anti-patterns and ultimately resulted in cheaper pricing due to better bin-packing. If you're doing K8s orchestration with Fargate, I highly recommend the switch.
(founder of WarpBuild - we offer hosted GHA runners)
This is a common issue.
A common pain point we see with users approaching some scale with self-hosting on k8s is that the k8s node autoscaling can become inefficient because of spiky loads.
We have a lot of users migrating off self-hosted setups using `actions-runner-controller` to ours because of this. Essentially, not having to deal with bin-packing is more efficient and concurrency, uptime guarantees are nice.
Sidecars are definitely a workaround but are hard to manage lifecycle for in conjunction with the primary container. This is now easier to do in 1.29+ with Sidecars officially supported via restartPolicy, but it was a colossal hassle prior to the advent of that. Also, we had noticed (maybe this has changed) that often logging frameworks were distro'd as DaemonSets (Helm, etc) and you'd have to shoe-horn the sidecar approach a bit.
