This makes sense. Once you have something with a lot of compute nodes, you have to orchestrate them all somehow. That's the real "operating system" for such systems. Each node just needs a loader and setup program to load work containers. Supercomputers, big cloud data centers, and things built from lots of GPUs tend to be organized like this.
Have things reached the point yet where containers run as Xen guests,
and the real OS is Kubernetes? Then you don't need Linux under your container, and with the right library, you don't need Linux in your container. Less OS state to get screwed up.
I feel like collectively the tech industry has forgotten what an OS is. We’re so used to Linux bring batteries included now that we assume we can supplant it.
Linux itself is not going away simply because replacing all those device drivers and ABIs is going to be incredibly difficult. Now, if you’re suggesting something new will sit atop Linux as the user land acting as a sandwich-filling between the Linux kernel and a multitude of userlands spanning many systems- I might agree.
>So when using POSIX, or any other programming model rich enough not to mess with OS specific APIs, the OS becomes irrelevant.
Except that things like systemd and a lot of redhat related software rely on Linux-specific functions like cgroups so they are not portable. Good luck replacing Linux even if something that much better comes up.
Not everyone makes use of those features and the last time I was using a Red-Hat system, it was for Java deployments, which is also not Linux related in any form.
Same applies to the C, C++, Fortran and now Chapel code that usually runs on HPC clusters.
Everyone uses systemd... I hope people are smart enough to not fall for it and not use all the scope-creeped features of it but if you add all the little things up, it turns out that we are getting more and more stuck with linux.
Leveraging Linux's driver support and resulting hypervisor independence was a major part of the rationale behind Xen, and a differentiator from VMWare, which has a strict HCL.
Linux is not the minimum viable OS. We have a lot of different situations where an OS can be much simpler, offering a limited set of functions or offloading multiple functionalities to different systems.
As pointed out before, mainframes have OSs that are exceedingly simple from the program's point of view and that offer a limited set of functions in exchange for extreme performance and stability.
Linux doesn't seem to be the bottleneck for neither performance nor stability in a lot of contemporary systems, though, particularly not if we're comparing against a new, hypothetical operating system.
Consider unikernels on VMs with paravirtualized hardware and thin hypervisors. When I boot my servers, hardware that's never used gets initialized. Filesystem code for filesystems that are never mounted is loaded. A whole lot of code that's specific to the CPU and chipset is patched in important execution flows. A unikernel has nothing but the code I want to run.
The fact we got it right most of the time, for most of the hardware combinations we use, does not make it any simpler, faster, or more reliable.
When I mentioned mainframe OSs, I was not talking about hypothetical new OSs but technology that has consistently excelled for decades now.
Have things reached the point yet where containers run as Xen guests, and the real OS is Kubernetes? Then you don't need Linux under your container, and with the right library, you don't need Linux in your container. Less OS state to get screwed up.