It's crazy and destructive that we are still using the unix paradigm in the cloud.
In the 70s we have transparent network fileststems, and by the 80s I had a more advanced cloud-native environment at PARC than is available today.* The Lispms were not quite as cloud native as that, but still you could have the impression that you just sat down at a terminal and had immediate window into an underlying "cloud" reality, with its degree of "nativeness" depending on the horsepower of the machine you were using. This is quite different from, say, a chromebook which is more like a remote terminal to a mainframe.
I was shocked when I encountered a Sun workstation: what an enormous step backwards. The damned thing even ran sendmail. Utterly the wrong paradigm, in many ways much worse than mainframe computing. Really we haven't traveled that far since those days. Cloud computing really still it "somebody else's computer."
There's no "OS" (in the philosophical sense) for treating remote resources truly abstractly, much less a hybrid local/remote. Applications have gone backwards to being PC-like silos. I feel like none of the decades of research in these areas is reflected in the commercial clouds, even though the people working there are smart and probably know that work well.
* Don't get me wrong: these environments only ran on what are small, slow machines by today's standards and mostly only ran on the LAN.
That is far, far more than transferring a byte of data. It's not the future though of course, this was well established in the 80s, it just so happens that it's a decent model for managing remote machines that outlasted over-engineered distributed designs like Plan9, VMS, MOSIX, etc. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like some vapid marketing pitch.
The great thing about Linux as the base layer is that it allows a commodity common ground with a very capable system that also facilitates more specialized layers to be implemented on top of it.
> The great thing about Linux as the base layer is that it allows a commodity common ground with a very capable system that also facilitates more specialized layers to be implemented on top of it.
The great thing about x86 instruction set as the base layer is that it allows a commodity common ground with a very capable system that also facilitates more specialized layers to be implemented on top of it. Only a handful of programmers develop at a layer below the x86 instruction set, or even at that level for that matter.
My point is that the instruction set of these large CPUs (not AVRs etc) is itself just an abstraction to an inscrutable micromachine. That abstraction lets you develop a complex program without knowing the details.
Unix was specifically designed for small, resource starved machines and did not have the powerful abstractions of mainframe OSes like Multics or OS/360. It's OK, but as modern CPUs and IO systems have grown and embraced the mainframe paradigms that had been omitted from the minicomputers (e.g. memory management, channel controllers, DMA, networking etc) unix and linux have bolted on support that doesn't always fit its own fundamental assumptions.
That's fine, it's how evolution works, but "cloud computing" is a different paradigm, and 99.99999% of developers should not have to be thinking at a unix level any more than they think of the micromachine (itself a program running on a lower level instruction set) that is interpreting the compiler's output at runtime.
As I said in the other comment, maybe people though that "cloud computing" is a different paradigm back in the 70's but it turns out that no, it's all the same distributed stuff.
If you have two processes on same machine, locking a shared data structure takes microseconds. You can easily update hundreds of shared maps and still provide great performance to user.
If you have datacenters in NY and Frankfurt, the ping is 90mS and fundamental speed of light limit say it will never be below 40mS.
So "lock a shared data structure" is completely out of the question, you need a different consistency model, remote-aware algorithms, local caches, and so on.
There are people who are continuously trying to replace Unix with completely new paradigms, like Unison [0].. but it is not really catching up, and I don't think it ever will. Physics is tough.
It's quite an appropriate analogy. We used to write a lot of code in assembly code. I used to write microcode and even modified a CPU. But it's been decades since I last wrote microcode (much less modified an already installed CPU!) and now the instruction sets of MPUs like x86 and ARM are mostly just abstractions over a micromachine that few people think about.
And an OS it the same: it used to be quite common to write for the bare iron, to which an OS is by definition an abstractional interface. I still do that, but it's an arcane skill frankly not in huge demand. Which is probably a good thing.
Nowadays most code is written at nosebleed levels of abstraction, which frankly is a good thing, even if I don't like doing it myself. But still, as developers do it, they are often dragged back down the stack to a level that these days few understand.
I think the person/company that cracks this will be the dominant infrastructure play of the decade.
It's not an appropriate analogy because most people don't program in x86, but most people know (or can easily look up, when the need arises) basic Linux administration commands.
That is far, far more than transferring a byte of data.
Educate me.
It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like some vapid marketing pitch.
That was a quick response to what I thought would be better than administrating a UNIX system on an Amazon machine. Don't be an ass. I've no experience with the real distributed computing systems created before UNIX.
The great thing about Linux as the base layer is that it allows a commodity common ground with a very capable system that also facilitates more specialized layers to be implemented on top of it.
No, that's the vapid marketing pitch. The Linux kernel randomly kills processes when it starts exhausting memory. It's garbage.
Hey, Brainfuck also facilitates more specialized layers to be implemented on top of it. Now, what's stupid about doing that, however?
Not sure if I'm being trolled... It finds a remote machine by name, and routes a connection to it. It authenticates and establishes a secure connection with the machine. It sends a command to the remote machine, the remote machine executes it, and the result is returned. It then puts the result into a form that can be used programmatically by the local shell.
> That was a quick response to what I thought would be better than administrating a UNIX system on an Amazon machine.
It was content-free.
> Don't be an ass.
What's good for the goose...
> I've no experience with the real distributed computing systems created before UNIX.
And already the expert. Impressive.
> No, that's the vapid marketing pitch.
No, that's the reality. That's why Amazon, Google, Azure, and everybody else offer it in their clouds and use it on their internal infrastructure.
> The Linux kernel randomly kills processes when it starts exhausting memory. It's garbage.
I'll take that over "running little functions floating in the void", it actually exists and works.
I'm being trolled by someone who finds a DNS lookup and an encrypted TCP connection to send a textual command to another machine is somehow impressive, rather than entirely basic and, again, unimpressive.
It then puts the result into a form that can be used programmatically by the local shell.
That does sound better than returns one octet with two well-defined values, sure.
And already the expert. Impressive.
I know UNIX is shit with a legion of cultists.
Hey, if we live in the best of all possible worlds, explain the market dominance of Windows. Did MicroSoft give Windows away for nothing, to poor unsuspecting university students who decided to hack on it instead of learn what a real computer is, until none remained?
It's even easier to shit on the status quo without having anything better.
> At least read my website before calling me unhinged.
No. I've seen countless tech prophets peddling snake oil over the decades, and it's always the same. Everything is dumb, poorly versed in the state of the art and history, but this magical ill-thought-out thing will somehow solve everything. It's very boring and predictable.
> No, that's the vapid marketing pitch. The Linux kernel randomly kills processes when it starts exhausting memory. It's garbage.
You sound like someone who wants some sort of magical fantasy abstraction level where errors never happen and you can just write code without ever caring about said errors or really anything that happens at a lower level or outside of your code. Sorry, but that doesn't exist and never will. Code runs on computers, it doesn't run in a "void".
Sure, you push the failure into the processes, and most OSes I've used do this. Thus (to use unix terminology) sbrk and fork() fail and it's the program's responsibility to handle that, gracefully or not as it wishes. You also degrade more slowly via paging.
You shouldn't get to the point where a perfectly innocent process is killed to reclaim memory. A process can die, or even (in some cases) recover cleanly from running out of memory -- that should be the process's choice.
In Linux you can do that -- disable overcommit (although I'm not sure if that's 100% foolproof). But that's often not preferred because that way does lead to "perfectly innocent" processes being killed: whoever allocates the next page when memory has run out loses. And how do they get to decide? What if a failure path requires paging in one more page of code from disk? Or causes a store to a COW page?
A boolean, a still (relatively) terse line of code. And when you end up wanting to do something even a bit more complicated it'll be a smooth gradient of difficulty.
As with every discussion around shells, though, the amazing terseness of scripts is great, but every form of programmatic behavior beyond "do this then that" and "do string interpolation here" is a massive pain in the butt. The biggest advantage is that stuff like pipes are a very good DSL.
What do you think happens when you invoke a shell? It's not some magical primitive, it also does string parsing and all the other stuff that a scripting language requires. SSH also has to parse the line!
I did say Python-esque (I do get that Python in particular has a lot of stuff in its prelude). Really if you want to be super pedantic then the most "banana"-y isn't a line in your shell but some C library calls on both sides of the system.
you're omitting the 500 lines of yaml needed to deploy that, the tens of megabyte for the python base image (of which you'll waste more than 99.99%) and the tens of megabytes in bandwidth to the docker registry and to download it on the kubernetes worker... And the image wight should be accounted at least four times: worker node + 3 in the docker registry (ha and stuff).
it's incredibly inefficient if you just freaking want to know if a file exists.
Am I understanding correctly that the concern is syntax based? If so, why not build a wrapper over the existing shell language? If not, can you please clarify?
Well my first thing is I would want a defacto norm adopted in the same way that bash is basically available everywhere. And of course we would want it to be something that doesn't have the busted dependency story of Python. And I would like for third party programs to have a rich interface be provided through this new shell.
For example, you can interact with docker in Python with a lib, but it would be amazing if the docker binary just exposed an RPC interface and API for script languages so you wouldn't have to parse out things from output or otherwise try to figure things out when writing shell scripts. This is, of course, the PowerShell thing, but PS's aesthetics displease me and many other people.
Computers are, of course, turing complete. You can do whatevery you want, really. But for example I end up using zsh instead of xonsh because I want to use what other people write as well.
Gee it's like nobody realizes that k8s is just unix machines too, only difference is instead of a program in the whole OS listening on a port, it's a container... And _gasp_ you can deploy containers to listen to any port you want and have those containers do anything you want... Wow.
I don't use k8s or whatever garbage gets thrown around nowadays.
I well remember my disgust when I learned AWS was just using UNIX virtual machines or whatever, rather than something simple, such as allowing people to run little functions floating in the void. I know nothing about AWS, but I've not been mistaken, right?
Well AWS has been quite clever, using both Xen originally and also now KVM VMs, but everything else is based on that yeah. But that's gotten really complex over the years. They now use a container execution environment called firecracker or something that is open sourced and might be totally separate from their virtualized environment - or at least on top of it where they do let you just run little jams. Cloud isn't bad, I just mean you do have to transfer octets somehow from exec to exec. There are more and less efficient ways of doing it, and also more and less secure ways too - and abstract ways. Not all bad.
Think the point is that as I am writing software, I don’t care about any of that. I expected an opinionated environment when I first heard the word cloud 2 or so decades ago; I wanted to put my code in cvs (git now) and that’s it; I don’t want to think of anything outside that. Especially security or scaling.
I ran 1000s of physical Linux servers over the decades using chroots when it was not fashionable to have containers yet with db clusters because I don’t want my laptop to be different than my servers and I don’t want to think or worry about scaling.
We have aws Lambda now, but it is too expensive and, for me, not trivial enough. It actually requires too much thinking (architecting) to make something complex with it and a lot of that is about money; you can stumble into bills that require at least a kidney to settle.
So I still run my own servers, these days with openfaas. With many scripts, it is almost painless and costs almost nothing (we provide for millions of users for a less than 200$ a month in total). But it still is not the dream: I still cannot just forget about everything hosting and abstractions still leak enough but at least I don’t spend more than a few hours a year on it and I don’t go bankrupt like can happen with Lambda.
We are building our own language and runtime (lisp like with types) to fix this: after 40 years of programming, I am pretty much fed up with having computing power and loving programming but having to deal with all this pointless (for programmers!) stuff around it: we went backwards, at least somewhat.
I like it for many reasons, but especially the helpful community, founder and everything just works. We use it with Istio at the moment. We switched from Cockroach (did not like the license and we had a few perf issues) to Yugabyte en Scylla recently for micro service and Monolith scaling to use with openfaas and it is really scaling to anything we need. Of course different situations have different needs, but this works very well for us.
well, when I first saw a aws prompt in 2006 or so, I was relief that EC2 was a plain debian-like machine.
Also I was saving me a phone call with Dell and a sunday evening configuring some hardware.
No new bullshit paradigm to learn, I could shove my code in there with tool I knew and call it a day.
EC2 was a stepping stone and it worked well to get some stuff in the cloud.
Remember the early days when only non-critical, non-PII stuff were in the cloud? And how some companies "just can't use it because of X" ( or worse the dreaded Y )
In the end, almost everything runs on some kind of OS. All those little functions in the void also have to run in an OS, but this is abstracted away from the user.
Congratulations on understanding what an abstraction is. Do I need to worry about the transistors or individual atoms in a computer? Why should UNIX be the right level of abstraction forever?
I didn't write to explain what an abstraction is, and I will assume you honestly meant to congratulate me instead of trying to be sarcastic. Thank you for that.
I have had two types of developers in my teams: one type that understands the underlying technology of the abstractions that they are using, and one type that only understands the layer/abstraction they are interacting with and blissfully clueless about anything underneath.
Neither group worries about transistors or proton decay in their hardware, and both can make 'things'. However, one group is much more capable of effectively building tools and understanding issues when something goes wrong in the chain, whereas the other group is regularly hopelessly lost because they have no clue of what is happening. I'll let you guess which is which.
This is not about worrying or even being bothered with details, but about understanding what you're working with.
such things however are just so often just so much better.
I was working with an intern lately, we both had remote machines in the same subnet (i'm omitting some details, of course), and he had to pass me a file.
We was about to download it on his laptop, upload it back to the cloud (slack message) when I would have had to download it on my laptop and send it back to the cloud.
I was able to show this young engineer how to use cat and nc (cat somefile.tgz | nc <ip-address> 8000) to send and then I would use nc (nc -l 8000 > somefile.tgz) to send it in a moment, without third parties involved and without having to make this file go across the globe multiple times.
The thing is: if you know what you actually need to do and have UNIX tools available, you can be insanely efficient.
"Treating remote resources truly abstractly" doesn't work in practice. Too many points of failure in our systems, and you really, really don't want to paper them over with abstraction if you want to build a fault-tolerant system.
That has been true for as long as I have been programming. But:
Similar statements have been made about high level programming languages. Nowadays most devs don’t understand how the CPU works, but write on the top of a tower of abstractions and nobody bats an eye. Many of those abstractions are quite complex!
I can imagine that the same could apply to certain kinds of network activities. Look at how ppl use http as a magical secure and robust inter process communication channel with no understanding of how it works at all.
Lambda is a half a baby step in this direction.
Another problem is tollbooths. The phone system uses a lot of bandwidth simply to charge the customer money. My phone company charges me overseas rates if I make a phone call outside the country, even if both endpoints are using WiFi, not the cellular network! I’m afraid of the same with fine-grained and abstract distributed computing, but perhaps the magical hand wave abstractions I posit above can help.
This tollbooth nightmare btw is the dream of the web3 bros.
We cannot afford the seamless distributed systems, and I don't think we ever will.
I use Python because I don't care if adding two numbers is taking a microsecond instead of a nanosecond. But if netwotk call suddenly takes 1 sec instead of 10mS? well, thats a huge problem, let's add a memory cache and rack-level cache and a parallel fetch and a whole bunch of monitoring.
The local compute is growing much faster than internet, and even faster than local network. I sure hope we get better abstractions, but caring about remote vs local call is not going away.
You could design a OS with abstractions built around latency, instead of physical machines. It would still allow you to find and use resources according to their constraints of use, but wouldn't force you to keep track on which exact machine they are located.
I am not sure what do you need new OS for, and what could you get from OS that you cannot get from today's computing.
If you want user to know they are talking to remote machine, but don't want them to care about which exact machine it is, we have a ton of great solutions already: load balancer, connection pools, service mesh, anycast, dynamic dns, etc...
If you want remote calls to be indistinguishable from local calls on the source code function
level, this is also solved! Many RPC frameworks and remote SDKs provide class-based interface which acts the same as the local class.
The only place where OS can help is if any OS function can be magically located on another machine. But even then.. we have remote filesystem (NFS), remote terminal and execution (ssh), remote graphics (x11), remote audio (alsa/pulse)... What is left for the new OS? process management? is it worth it?
> If you want user to know they are talking to remote machine, but don't want them to care about which exact machine it is, we have a ton of great solutions already: load balancer, connection pools, service mesh, anycast, dynamic dns, etc...
Those resources are complex to program against. An OS should offer a simplified abstraction layer to make them as transparent as possible. And yes, process management is worth having a unified programming model that doesn't force you to keep track of where each process instance is being located - that's essential for massively parallel computing.
Of could this could be done with platforms for massively parallel computing. The point of building an OS would be to put these platforms as close to the metal as possible to improve their efficiency.
The paradigm for abstracting away a thousand CPUs is AWS Lambda/GCP or Azure functions/K8s' implementation of serverless. It's not a total drop in replacement because a plain lift-and-shift can't change your paradigm, but cloud functions are very much a Cloud 2.0 (or at least 1.5) paradigm.
S3, yes a network accessible FS.
Unix - only is a telling word - it is the /defining/ paradigm for that. Is there a better one yet?
1000 CPUs? I mean, uh Hadoop, spark, etc etc. What?
> Unix - only is a telling word - it is the /defining/ paradigm for that.
Don’t be ridiculous. We had networked distributed file systems over both the arpanet and lans before Unix even had networking. I even mentioned this in my root comment.
Unix did make it work much better a bit later, organizations used to run NFS and single sign on with Kerberos to workstations, automated from scratch reprovisioning of workstations (so you could just reinstall from tftp server with org specific SW customizations included if previous user had messed the box up), smooth remote access to all machines including gui apps, etc.
It just went away due to Microsoft crowding it out.
Mainframes are very expensive. You can buy mainframe with very fast CPU and RAM interconnect and scale it by buying more hardware. Or you can spend 100x less and buy a number of server blades. Interconnect will be very slow, so you can't just run some kind of abstracted OS, you need to run separate OS on every server blade, you need to design your software with that slow interconnect in mind. But in the end it's still 100x cheaper and it's worth it.
Also mainframes have growth limit. You can buy very powerful ones, but you can't buy one that's as powerful as entire datacenter of server blades.
That's why I both hope Oxide Computers succeed and worry they may not.
They are effectively building a mini computer. The smallest unit you can buy from them is an entire rack. Modified rackmount hardware with better software to make it more cohesive.
I really hope they go to half-racks, but I've no idea how you'd stack them.
I'm sort of reminded of how the US government is the worst (except for all the rest), when having an absolute ruler should be so MUCH more efficient. Problems would be fixed by fiat.
Or maybe, why does lisp persist with its horrible user-unfriendly syntax?
:)
I guess we will just have to invent it. (and you should do your part by reminding people with examples of old systems that elegantly solved the papercuts of today)
That’s a hand wavey way to make a claim that can’t be backed up.
What you had then was in no measurable way more advanced architecturally or conceptually. Name one facet in which it was more, could do more, or faster, or better.
You can’t because it couldn’t. No part of your setup was cloud native. Nothing was abstracted away, a core tenant of the cloud.
Abstractions usually (always?) have a cost because physics.
> The damned thing even ran sendmail
and?
> Cloud computing really still it "somebody else's computer."
That's the definition of 'the cloud'. Unless you run it locally in which case it's your computer. What's your point.
> There's no "OS" (in the philosophical sense) for treating remote resources truly abstractly
It's unclear what you're asking for. Treating stuff truly abstractly is going to get you appalling and horribly variable scalability. If you're aware of that, why don't you tell us what you want to see instead.
Edit: ok, this is from gumby, now I recognise the name. This guy actually knows what he's talking about, so please tell us what things you would like to see implemented.
>> Cloud computing really still it "somebody else's computer."
> That's the definition of 'the cloud'. Unless you run it locally in which case it's your computer.
Forget the stupid framing of idiotic marketers in the early 00s and go back to the original “cloud” definition (that engineers were still using in those ‘00s but was distorted for a buck).
The term was introduced (by Vince Cerf perhaps) in the original Internet protocol papers, literally with a picture of a cloud with devices at the edge. It was one of the revolutionary paridigms of IP: you push a packet into the cloud (network) but don’t need to / can’t look into it and the network worries about how to route the picket — on a per-packet basis! You don’t say “the cloud is other peoples’ routers”.
Today’s approach to remote computing requires developers to know too much about the remote environment. It’s like the bad old days of having to know the route to connect to another host.
You don’t pay per packet even though a huge amount of computation is done on devices between you and the machine you’re connecting to in order to transmit each one.
When you emerge from the jungle, you may notice that not only UNIX conquered the world but even ""worse"" paradigms of Windows and iOS have proliferated. You have to ask why the situation that is so much worse is so popular: is it really everyone else who is wrong?
Appeal to the currently incumbent solution is not convincing. Very often the majority of people simply choose the lesser evil -- not what's of the best quality or with the biggest productivity.
Or need I remind you that hangings at sunrise and sunset were commonplace and people even brought their kids to them?
I'm sure back then people defended it as well, and it's likely that if you heard their arguments you'd facepalm.
I did. Its use of intentionally obscure language that makes APL seem readable and consistent in comparison just because "only smart folks should be able to code in this" is something I simply can't accept. And I love obscure langauges!
Good replies by others here. "crazy and destructive" that you have no idea about how computers work today, or how computers are still computers. Your ignorance about things like Sun workstations as it relates to literally everything today, I mean you have no idea about modern computing lol
In the 70s we have transparent network fileststems, and by the 80s I had a more advanced cloud-native environment at PARC than is available today.* The Lispms were not quite as cloud native as that, but still you could have the impression that you just sat down at a terminal and had immediate window into an underlying "cloud" reality, with its degree of "nativeness" depending on the horsepower of the machine you were using. This is quite different from, say, a chromebook which is more like a remote terminal to a mainframe.
I was shocked when I encountered a Sun workstation: what an enormous step backwards. The damned thing even ran sendmail. Utterly the wrong paradigm, in many ways much worse than mainframe computing. Really we haven't traveled that far since those days. Cloud computing really still it "somebody else's computer."
There's no "OS" (in the philosophical sense) for treating remote resources truly abstractly, much less a hybrid local/remote. Applications have gone backwards to being PC-like silos. I feel like none of the decades of research in these areas is reflected in the commercial clouds, even though the people working there are smart and probably know that work well.
* Don't get me wrong: these environments only ran on what are small, slow machines by today's standards and mostly only ran on the LAN.