A standard in the Linux world. Windows doesn’t have “distributions”, so there’s less need for containers in that space. Microsoft uses Service Fabric internally for the Azure back-end, which covers most of the other use-cases of Kubernetes.
Service Fabric needs to die. It is an abomination on software.
Why, why, *WHY* does each SF service need to have some terrible wrapper code that forces it to run in, and only in, SF? Oh ya.. vendor lock-in.
No thanks.
Yes, I'm aware that MS added support for docker containers in SF. That integration is dreadful. They did about as good a job of it as the WCF team did of shoe-horning "RESTFul Services" in.
> Windows doesn’t have “distributions”, so there’s less need for containers in that space.
I don't follow. What does distributions have to do with need for containers?
> What does distributions have to do with need for containers?
A lot (but not everything).
It's important to recognise where the "stable API/ABI boundaries" are. In Linux, the stable boundary is the kernel. Userspace is unstable. There are many distributions, with many small and big differences. Different C runtimes, different folder structures, different configs, modules, etc...
In Windows, the stable boundary is the Win32, COM, and .NET APIs, which are in user space. The kernel boundary is not stable[1] however, which also matters.
Even Linus Torvalds has trouble distributing his hobby dive-computer software to Linux![2] He has no issues with MacOS and Windows, because they have stable APIs and distribution mechanisms. With Linux, the kernel is stable, but essentially nothing else is. If you want to distribute software to everyone, then this is a complex problem with many moving parts that you have to manage yourself.
Containers sidestep this by packaging up the distribution along with the software being deployed. This works because the kernel ABI is stable.
[1] In the Windows world you don't really need to package up the "Windows distribution" because there is only one: Microsoft Windows. Conversely, Windows containers aren't truly portable because the Windows kernel ABI isn't stable. However Microsoft changed this with Windows 11 and Server 2022, the kernel ABI is now "stable", or at least sufficiently stable for Server Core containers to be portable for 5+ years as-is.
[2] He's complaining about "desktop applications", but the exact same rant would apply to server software also. This one talk made me understand containers and why they're so important in the Linux world: https://www.youtube.com/watch?v=Pzl1B7nB9Kc
Even if Windows containers are not as portable, the ability to manage them from the same Kubernetes controlplane that manages Linux containers seems pretty powerful. It also enables you to do it outside of Azure/Service Fabric on other clouds or metal.
You're trading one convenience for a world of hurt if you use Kubernetes with Windows. Just take a casual glance at the "Windows containers" issues list on GitHub: https://github.com/microsoft/Windows-Containers/issues
Some of the very recently closed issues were jaw-dropping, such as totally broken networking in common scenarios.
DNS resolution is very different in Windows compared to Linux, making much of the "neatness" of Kubernetes pod-to-pod communication not work.
There is no maximum memory limit in Windows kernel "job objects" (equivalent to cgroups), so one memory leak in one pod can kill an entire node or cluster. This is very hard to solve, and I've seen it take out Service Fabric clusters also.
Thanks for sharing, it all seems terrifying but still very interesting to explore. I was curious how the Windows nodes would even join - turns out they support cloud-init just like Linux machines.
Not so much, no. Linux is the kernel, and all Linux distros, every single one, employs Linux as its kernel, or else it would not be Linux. Stepping back from my pedantry, I believe what you must mean is that different Linux distributions will customize the kernel a bit. That doesn't change what it is. Just because you have automatic windows and I have cranks, and you have leather interior and I have plush doesn't mean our vehicles aren't the exact same year, make and model.
Sure, I can imagine, but that is merely a bug, or perhaps the compatibility issue stems from misuse of a general but inaccurate claim of "Linux compatible." My expectation is specific compatibility is stated plainly in the documentation and on the box, if there is one, without ambiguity.
Note that because of Linus's strict policies on userspace compatibility, A>B is basically guaranteed to always be safe, and only A<B will ever cause problems, unless distros add patches that break things.
I e heard this before, but the only case I’ve heard of people running into issues is if you hit a kernel bug or depend on a really new feature that’s not present in the kernel yet.
Mixed workloads. Not everything inside MS is Windows, in fact, more than people probably realize is running on Linux. We have our own internal Linux distribution (Mariner) and own Kinvolk (which maintained CoreOS/Container Linux/Flatcar Linux).
