Things don't have to be the same to compare & contrast. There were dozens of articles a month comparing vm's Vs containers back in the day.
I also think your fixed perspective that these are & always will be different levels of the stack is an inadequate fixedness, lacks the mental agility to comprehend many of the things wasm is growing into. For years already, standard libraries like wasi are attempting to become fairly complete platform libraries. Instances of wasm run in effectively process trees of chroots, so the simar parity is apparent.
It's absolutely coom sense to be comparing startup times across the two. There have been endless efforts to try to make cold boot containers or container snapshots as fast as possible. And this again extends backwards in time to comparisons versus vm's as well, with efforts like the Xen Zerg demo trying to make vm's fast enough to boot a vm per request. https://news.ycombinator.com/item?id=5243360
Getting back to a process per request or process per session model is a holy grail, one that every virtualization/containerization technology has explored. It would greatly alter the application development model & potentially greatly enhance security, could help radically reshape resource management. This article is doing a fine job pointing to the future, to what we are unlocking. We absolutely should compare versus what the technology of today (containers) can do. The comparison is spot on.
> WASM vs a container runtime isn't a meaningful comparison because they're at different layers of the stack. A container runtime runs a process tree in a chroot, and whether the process binaries are native code (x86, ARM) or bytecode (JVM, WASM, CLR) is an implementation detail of the runtime.
It's a meaningful comparison because they often solve the same business problem; you need to run some code in an isolated/managed environment. Whether that's done through a process tree or a single process is an implementation detail for many purposes; not being able to run multiple processes might have implications for some use cases, but a lot of systems (e.g. docker/kubernetes, or especially anything serverless) are oriented around using a container for a single process.
A comparison between WASM and x86/ARM/etc native binaries is meaningful, because those are different formats for executable code.
A comparison between WASM and "containers" is meaningless, because the binaries being executed by a container runtime could be written in WASM. The executable format and the packaging format are separate concerns.
This might be easier to think about if you imagine the article was written as "native code vs containers" -- what conclusions (if any) could you draw from the benchmarks in an article with that premise?
A comparison between WASM and "containers" is meaningful because in popular usage there's a cottage industry of WASM-based container solutions with the express goal (and co-evolved with the language to make this possible) of simplifying some of the thornier design decisions that make other containerization solutions easier to get started with (can just drop in any binary and dependencies) but also less performant for some set of tasks (have to set up network cgroups and other garbage). The phrase "WASM" isn't just being used to refer to the actual stack machine.
If someone wants to try to advertise some weird custom platform that uses WASM as an intermediate bytecode for its packaging format then that's fine, but they'll need to pick a different name.
WASI is a part of the WASM charter, and parts of WASM are explicitly designed to be friendly to such ideas; It's not quite _just_ some choice of bytecode.
Besides, words are just approximations of thoughts, and even on the best of days the delineations aren't crisp. There's a critical mass of people using WASM+containers to refer to WASM ... plus containers (especially those built on the core tech supported by the WASM mission -- WASI -- like the platform tested in the article).
Your argument is nonsense. It's like arguing that comparing the performance of an API that use lambdas VS a traditional API with an application running in the server is meaningless because "the lambda could be written in the application server". Who cares? If both can be used to implement the same API, the comparison is meaningful.
Ok then. Let's say I write a webserver with WASI, compile it to a `.wasm` file, package it as an OCI container image, and deploy it in Kubernetes via a container runtime that loads the entry point into wasmtime.
If containers and WASM are in fact competing technologies, as you and the article imply, then this shouldn't be possible.
On the other hand, if WASM is a format for executable code as I claim, then packaging WASM as a container image and executing it in a container runtime is fine and should work as well as any other machine-independent bytecode (such as JVM).
WASM is a bytecode format like Java bytecode... the difference is that it's sandboxed. It can't do anything that's not explicitly given to it, including using WASI (the POSIX-like API for WASM code to "talk" to the system).
Because it's sandboxed, you can use it without a container. I think that's what you don't seem to undestand. There are several companies offering WASM-based "containers" in the cloud - which don't need any actual container, the WASM runtime has the same type of capability of a container, to isolate processes running on the same hardware securely.
All bytecode is sandboxed by definition -- the VM decides which operations it delegates to the system. It is possible (and common!) to use JVM bytecode in a VM that restricts access to the underlying system, and in fact one of the original core use cases for Java (applets) relied on this functionality.
The difference between WASM and JVM is that WASM was designed to be both (1) a compilation target for memory-unsafe languages like C, and (2) have verifiable control flow so it's easier to write a provably-secure JIT.
> Because it's sandboxed, you can use it without a container. I think
> that's what you don't seem to undestand.
You don't seem to understand what a container is, or what a sandbox is.
The reason to deploy software as a container image (instead of a standalone binary) is that a container image contains a full chroot, including auxiliary files that aren't embedded into the main binary. Whether the binary is native machine code or WASM doesn't matter -- the tradeoffs of bundling everything into one binary vs placing them on disk are the same regardless of executable format.
The reason to run a sandbox is to restrict which system resources a process can access or interfere with. You can sandbox a process whether or not it runs in a container.
I see you like to ignore reality and go on with your own definitions for things.
> All bytecode is sandboxed by definition
Oh my... you probably mean "all bytecode COULD be sandboxed"... try sandoboxing JVM bytecode... they tried that for years :D didn't quite work.
> The reason to deploy software as a container image (instead of a standalone binary) is that a container image contains a full chroot,
No. Stop trying to tell us what a container should be used for.
A container's main usage nowadays is to run multiple isolated "processes" or set of processes in a safe manner, making the most of the hardware... coincidentally, you can also do that using WASM. Get over your outdated worldview, mate.
EDIT: please read the link I provided. From the creators of Docker:
"If WASM+WASI existed in 2008, we wouldn't have needed to create Docker."
How does that align with your definition of what a container is or isn't?
As an aside, to what degree does WASI exist _now_? The spec seems to be in a state of flux as to what the language the spec is written in is, and the language new proposals are written in seems to have grown futures? On the other hand, as far as I can tell, the latest draft spec doesn't look to have support for UDP or scalable notification (epoll, kqueue, etc.), so is that really what people are using their existing software with?
Docker's great win is that you don't even need to recompile your software to use it, much less port it to what's essentially a new non-POSIX operating system.
> As an aside, to what degree does WASI exist _now_?
It's ... rough. The impression I get (as an uninvolved third-party) is that the WASI folks set out with a goal of writing a sort of capability-based POSIX that they could write a thin libc for, but partway through realized that POSIX embeds a lot of assumptions that aren't compatible with portability or capability-based security.
IMO (again, no direct involvement) a better approach would be to start with a low-level API that is capability-based from the beginning. Something like EFI, or (more experimentally) an RPC-oriented syscall API resembling Fuchsia/Plan9. Then the various sorts of magic integers and process state found in POSIX can be layered on top of that as a library.
The good news is that the people working on WASI do seem aware of what they've gotten themselves into, and I think a lot of the chaos (shifting repos, broken docs, etc) is just the natural outcome of them exploring the solution space. My expectation is that they'll be forced to drift pretty far from POSIX if they want a workable design, but maybe they'll surprise us.
> If containers and WASM are in fact competing technologies, as you and the article imply, then this shouldn't be possible.
That doesn't make any sense to me. Just because you can put a thing inside another thing doesn't mean they don't also solve an overlapping set of problems.
Let's replace with food.
Comparing eating soup with eating at a restaurant is meaningul because both are used to solve the same problem. Arguing that you can also eat soup at restaurant so these things are at different stack layers and should not be directly compared is a nonsense argument.
The article isn't about WASM vs a container runtime, it's about comparing two container runtimes.
WasmEdge is a container runtime designed for running Webassembly
If you check the whole article, you'll find the (simple and basic) benchmark comparing a rust "hello world" running with runc, and (apparently) the same code compiled to Wasm and running in WasmEdge.
>If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!
> It's not just comparing pure WASM (bytecode) but the WASM+WASI (runtime)
WASI isn't a runtime, and it's not a packaging format. It's a standard ABI for a POSIX-like syscall interface, which is useful[0] but doesn't substitute for or relate to container runtimes.
You still need a syscall shim to provide the WASI implementation, and you need a WASM VM to run it.
> It's a meaningful comparison when the original creator of Docker
> containers also compared them:
I've seen that quote before and I think it's nonsense.
Docker container images are architecture-specific, and it took years before their distribution protocol supported specifying the architecture. If the existence of an architecture-agnostic bytecode would have avoided the need for Docker, then the JVM would have filled that niche perfectly.
And if you look at the Docker code, the vast majority is about configuring the runtime environment of the container (networking, bind-mounts, logging) and persisting that state across reboots. There is no syscall shimming like in gVisor or a WASI implementation, it's all about setting up a chroot and then calling `exec()`.
If WASM had existed in 2008, maybe the Docker folks would have enabled it as an entry-point format, but you'd still need a way to ship all the other stuff that goes along with the main binary.
It's baffling that people feel the need to fabricate ways for WASM/WASI to be important, when the projects themselves are already so interesting. Architecture-agnostic plugins! A clean slate for a capability-based syscall ABI! A genuine chance at running unmodified userland programs in memory-safe sandboxes! Get excited about that, not about someone poorly re-implementing AWS Lambda.
[0] Maybe one day most small shell tools will be in WASM+WASI, and the typical Linux distribution will have relatively little native code installed by default.
Actually there is support for using WASM as a container runtime. You could in theory have WASM containers and Linux containers next to each other on the same K8s cluster. Not sure that I’d recommend it in production just yet, but it’s certainly usable
I think your comparison to CGI is unfair, since CGI in containers is quite different from "classic" CGI imo.
Wikipedia describes CGI as "In computing, Common Gateway Interface (CGI) is an interface specification that enables web servers to execute an external program, typically to process user requests.[1]
Such programs are often written in a scripting language and are commonly referred to as CGI scripts, but they may include compiled programs.[2]
A typical use case occurs when a web user submits a web form on a web page that uses CGI. The form's data is sent to the web server within an HTTP request with a URL denoting a CGI script. The web server then launches the CGI script in a new computer process, passing the form data to it. The output of the CGI script, usually in the form of HTML, is returned by the script to the Web server, and the server relays it back to the browser as its response to the browser's request.[3]"
Is it possible you don't have direct experience with CGI?
The typical way to deploy CGI during the '90s was to have a compiled program in C/C++, which would be executed once per request. There were entire frameworks built around this "process per request" model, with various optimizations like pre-warming (start the process, connect to db, wait for request on stdin) and variant protocols (SCGI).
Any time you have an execution model based on running a single address space per request, it's fundamentally CGI-ish. It doesn't have anything to do with the implementation language. The limitation of CGI is that state can't be persisted in-process between requests.
---
Going back to the thread, there's multiple aspects when thinking about the structure of a hosted service:
1. Whether state is shared between requests (CGI vs long-lived process).
2. Whether the binary is native code (x86/ARM/) or bytecode (WASM/JVM/).
3. Whether the binary is standalone or comes with additional files.
4. If it needs additional files, whether it's distributed as a package (apt/rpm) or chroot (container image).
Does breaking those different concepts down into a list help? You can see that trying to compare the performance of "WASM deployed as standalone binary with CGI execution model" and "native code deployed in container image" just isn't meaningful. The article is mixing up too many different ideas when trying to benchmark.
I ran PHP in CGI mode a long time ago if that counts :-)
I think what the article is implicitly doing is comparing WASM bytecode running under a container runtime that executes it directly (e.g., without a full chroot). Your point, iiuc, is that there are many variables here, of which x86 vs WASM is only one, and that the benefits here might not be due so much to WASM so much as they are due to the other factors.
I didn't understand where CGI-ness comes into it though, as I don't think there's any difference between this kind of container and any other in terms of the duration or when it's starter up, but perhaps your point is that it's lacking some "full" environment, which makes is akin to CGI? I guess a more accurate comparison would be against an x86 binary being executed with some kind of minimal docker runtime, then? I guess the article could be interpreted generously then as a comparison between a thin docker runtime and a thick docker runtime.
I guess that across the board vendors are experimenting and evaluating the impact of introducing WASM/WASI-based technologies and that may constitute something of a paradigm shift.
On the container side Docker also has native WASM support in a technical preview [0]. I don't think they see WASM as a threat, but rather as an addition to how apps and services can be delivered.
The other day I found an InfoWorld article "Solving the SBOM crisis with WebAssembly Components" [1] that features a diagram on the evolution of Application Development Stacks (by Cosmonic, one early adopter Wasm vendor) that indicates the trend. Idea is to tackle some of the complexity in current infra setups that have become like "Rube Goldberg machines".
Right now most early examples alas boot a container with a wasm runtime for each wasm instance, which is a sad waste. The whole advantage of wasm should be very lightweight low overhead wasm runtime instances atop a common wasm process. Having a process or container for each instance loses a ton of the benefit, makes it not much better than a regular container.
It's still being used to spawn a wasm processes per instance for now, but container runtime project Kuasar is already using the Sandbox API to save significant resources, and has already chimed in in comments on HN to express a desire to have shared-process/multi-wasm-instamxe runtimes, which could indeed allow sub ms spawning that could enable instance per request architectures. https://github.com/kuasar-io/kuasar
"Nobody is deploying large-scale services on top of what would be essentially the CGI model." Yes, they are. That's basically the Cloudflare workers model (which certainly qualifies as large-scale) and many other V8/Wasm based serverless solutions
The article is not describing the Cloudflare approach (v8 isolates) -- note the graph of startup times. Their "wasmedge" line takes 20 seconds to start 1500 instances, or about 13 milliseconds per instance.
The author has carried out an assessment of how the use of the ARM assisted hardware virtualisation (normally used to isolate and run VM's but not containers) affects different metrics of running different runtimes in a container environment that is aware of and that can take advantage of the hardware virtualisation features:
Kata Container is a secure container runtime that uses hardware virtualization technology as an additional layer of defense. This allows it to provide stronger workload isolation while still being lightweight and performing like regular containers.
The author uses WASM runtimes as a conduit, and concludes the article with a compilation of results and the impact on each metric, namely: boot times, security, the ease of development in each setup, production readiness amongst others. The author does compare raw computational performance of each option, and it does not appear to be an objective.
The article title could have been better worded, though.
