I mean.. I appreciate that there are proponents and people trying to improve the state of async rust but to allude that everything is dandy is either dishonest or more likely a strong curse of knowledge bias.
I’ve worked deeply in an async rust codebase at a FAANG company. The vast majority chooses a dialect of async Rust which involves arcs, mutices, boxing etc everywhere, not to mention the giant dep tree of crates to do even menial things. The ones who try to use proper lifetimes etc are haunted by the compiler and give up after enough suffering.
Async was an extremely impressive demo that got partially accepted before knowing the implications. The remaining 10% turned out to be orders of magnitude more complex. (If you disagree, try to explain pin projection in simple terms.) The damage to the ecosystem from fragmentation is massive.
Look, maybe it was correct to skip green threads. But the layer of abstraction for async is too invasive. It would have been better to create a “runtime backend” contract - default would be the same as sync rust today (ie syscalls, threads, atomic ops etc – I mean it’s already half way there except it’s a bunch of conditional compilation for different targets). Then, alternative runtimes could have been built independently and plugged in without changing a line of code, it’d be all behind the scenes. We could have simple single-threaded concurrent runtimes for embedded and maybe wasm. Work stealing runtimes for web servers, and so on.
I’m not saying it would be easy or solve all use-cases on a short time scale with this approach. But I do believe it would have been possible, and better for both the runtime geeks and much better for the average user.
Pin projection is the proccess of getting a pinned reference to a struct's field from a pinned reference to the whole struct. Simple concept, but the APIs currently on offer for it (`unsafe` code or macro hackery) are very subpar.
Yeah, perhaps. But I am not part of that minuscule subset of people who have deep expertise in both compiler internals and runtime architecture to have well founded opinion on the design. There is FFI and stack issues that’d need some incredibly bright engineers to sort out.
My argument is more along the lines of: modularity is the (only) way to reduce complexity. We already have modular runtimes in other languages (project loom in Java, webassembly etc). Most people should not care about runtimes much. The ecosystem cost of async ended up being high. Thus, runtimes should be an implementation detail for most users.
Doesn’t mean Rome has to be rebuilt. Perhaps the async we have can be saved, but even so it involves biting the apple of actually defining precisely what a runtime is so that crate authors can think of them just like they think of allocators today (ie not at all).
I do tend to agree with you, but just to note that both the approaches you listed for this are more recent than the decisions rust made on this. It's not "good approaches to modular runtimes were already rock solid, why didn't they consider them?". It's "people have done promising work on this in the last decade, maybe rust could figure out how to incorporate it in some way moving forward".
Oh for sure. Armchair pointing in hindsight is trivial, or at least easy. The folks who fleshed this out at such an early stage did an extremely impressive job.
What an amazing article. I can't believe I missed it when he wrote it.
I followed Rust in the very early days and definitely came away with the sense in this article. I would have said (and may have said to some people) that Graydon is really great, but that the exciting things about Rust weren't the things he liked or cared about; basically the expressivity and zero cost abstractions sections of this article.
But reading the article he linked about first class modules, I think that seems pretty good, and I think he's definitely right about making borrowing "second class" without explicit lifetimes (or at least discouraging them more so than the language does today), and about existential types (I'm always surprised I don't see these more in library APIs).
I also had no idea he wanted built in bignums. In pre-1.0 (and pre-cargo) rust, I created a very incomplete library for that, and would have loved to have it built in instead. Also yeah, decimal literals would be excellent.
But I didn't find the async vs. green threads section convincing. The green thread implementation wasn't a great fit at the time it existed, and I haven't seen anything since then that convinces me there was some great solution available to make it work better. Async isn't great in rust, but it's a much better fit, and I think it can be used well. I have hopes that best practices developing over time and maybe language features or changes can push people in a more sane direction of usage (once it becomes more clear what that should be).
I don't have a degree in CS so I always feel out of my element in these discussions...
Is the runtime something the compiler adds to the binary to make sure it is able to correctly interact with the system it is built for?
It seems like people argue that green threads require a runtime as if async doesn't? I don't understand the arguments on either side. In terms of what code looks like I far prefer being able to just declare green threads like golang does.
Honestly I wish I understood on a deep level, but I've been programming for 17+ years and the fact that I still don't implies to me that I never will.
A Runtime generally refers to "things added to the program to make it work". That can mean libc, it can mean a GC, etc.
> I don't understand the arguments on either side. In terms of what code looks like I far prefer being able to just declare green threads like golang does.
Under the hood `async` is sugar over a function such that the function returns a `Future<T>` instead of a `T`. What is done with that future is up to the caller.
In most cases this is handed off to a runtime (your choice of runtime, generally speaking) that will figure out how to execute it. You could also manually poll the future until it's complete, which does happen sometimes if you're manually implementing the Future trait.
If you have no async code you can simply avoid having an async runtime altogether, reducing the required runtime for an arbitrary program.
> I far prefer being able to just declare green threads like golang does
This relies on an implicit runtime. That's fine - lots of Rust libraries that work the way you're suggesting will just assume a runtime exists.
That lets you write:
spawn(async {println!("hello from async");});
And, just like a goroutine, it will be scheduled for execution by the implicit runtime (or it will panic if that runtime is not there).
Note that this implicit runtime has to be there or you'll panic. This means that the reasonable behavior would be to always provide such a runtime, which would mean that even "sync" programs would need it. Or otherwise you'd need to somehow determine that no "async" code is ever actually called and statically remove it. That is a major reason why you wouldn't want this model in a language that tries to minimize its runtime.
> but I've been programming for 17+ years and the fact that I still don't implies to me that I never will.
I think it's just a matter of exposure. Try writing in more languages like C, C++, Rust, etc, and dig into these features.
That's what I'm saying. When a library uses "spawn" it is generally assuming a runtime, typically tokio (although in my experience a lot of libraries are generic).
They aren't different. They are the same. The only difference is that in Go the `yield` points are implicit (the compiler inserts them) whereas in Rust they are explicit (.await).
Otherwise they both schedule a task to be executed with the implicit runtime - in Rust that may be tokio or something else, in Go that would be the Go runtime.
I would say they function similarly. There are quite a few internal differences. The main benefit of Rust is that futures are just regular types and don’t need a stack when idle. And the main benefit of Go is that there is only one type of function, so no coloring.
> they both schedule a task to be executed with the implicit runtime
To be pedantic, in rust the runtime is referenced with a global or thread-local variable, but it’s still explicit. This means crate authors can’t spawn tasks without depending on a runtime… unless there’s been recent developments.
Sure, their implementation differs in all sorts of ways. But for the purposes of this conversation, with regards to scheduling a future without having to manually pin it up into a your call graph, they are the same.
> but it’s still explicit.
I guess it just comes down to your definition of explicit. There's a dependency, but from a caller's perspective it's implicit. It doesn't matter though, I think the point is clear enough.
I generally agree, but specific to your point about arcs/mutexes - what would be the alternative for shared mutable data? Or do you mean people use it for stuff that isn't shared too?
> Or do you mean people use it for stuff that isn't shared too?
Exactly. RAII works beautifully in regular Rust, so you create references with the static ownership rules and pass them around, before the value is dropped at a deterministic place. This is like the main value prop of Rust.
In async Rust OTOH (in fact regular threads as well) it’s much harder to use references when they normally would make sense. So instead of `&T` and `&mut T` you need `Arc<T>` and `Arc<Mutex<T>>`, respectively.
Then you lose both on performance (the initial blog post claimed that the pervasive arcing is worse than GC) but also UX. Arcs are much easier to leak, for instance.
> The vast majority chooses a dialect of async Rust which involves arcs, mutices, boxing etc everywhere
And?
> not to mention the giant dep tree of crates to do even menial things.
Again, And?
I don't really care about having to pull in crates. That has always been how Rust does things - it prefers many small crates over fewer large crates. Async is no different.
And I don't care about Arc either. Writing `let x = blah()` is not much better than `let x = Arc::new(blah())`.
If you're talking about something else, like idk, maintaining mutability across multiple threads, yeah that's going to be more painful. It's also painful in most other languages and is generally avoided for that reason.
> The ones who try to use proper lifetimes etc are haunted by the compiler and give up after enough suffering.
You say "proper lifetimes" as if lifetimes are desirable. In async code they are not - your lifetime is often "arbitrary" and that's what an Arc gives you. The solution is, as mentioned, using an Arc or Box or Mutex.
> Async was an extremely impressive demo that got partially accepted before knowing the implications.
I think this is a totally ignorant characterization of async, which was years in the making, took lessons learned from decades of async in other languages, and was frankly led by some of the most knowledgeable people in regards to these sorts of systems.
> (If you disagree, try to explain pin projection in simple terms.)
The vast majority of people will never have to know what a pin projection is, let alone how it works. It rarely comes up, and virtually only if you're writing libraries. I could explain it but I see no reason to do so here (it is not complicated at all, `Pin` is probably the harder one to explain).
> The damage to the ecosystem from fragmentation is massive.
It's not even noticeable lol like, what? What fragmentation? I've never run into an issue of fragmentation and I've written 100s of thousands of lines of Rust.
> It would have been better to
How nice to sit on the sidelines and throw out a paragraph sized proposal. Everything looks great when you hand wave away the complexity of the problem space.
Async Rust isn't perfect (I frankly don't think there is a "perfect" solution, that should not be contentious I hope) and I welcome criticism, but your post is totally unconstructive and unsubstantial.
I am of the same impression — Rust as a language forces you to come at terms with your own ideas of how code should look.
That being said I still have a deep dislike of having to read through nested Arc Mutexes or whatever to figure out what the code does in principle before I figure out what is going on in detail with the ownership.
I know there are no perfect solutions and there are trade-offs to be made, but I wish there was a way to have it more readable.
So instead of this:
let s = Arc::new(Mutex::new(Something::new("foo")));
FWIW Arc implements From<T> so you can do `let foo = blah().into()` and if `foo` is passed to something expecting Arc it'll be an Arc. This probably works for Arc<Mutex<T>> as well so you'd be able to do `.into().into()` but I'm not sure.
Iirc you can't chain .into() because the compiler can't be sure whether the first is meant to do the whole conversion and the second is the identity (impl From<T> for T), or if the first does one step and the second the next or if the first is the identity and the second does the whole conversion. (In fact I think the existence of the first and third options alone is enough to preclude chained .into() from working due to the ambiguity.)
I might need a lot of stuff in my software. Eventually. I might need distributed database, or to to scale it out to run on multiple machines, and then maybe Raft, or reactive architecture, zero-copy IO, or incremental updates or ... or ... the list goes on and on.
Thinking too much and in particularly going with over complicated solutions from the very start because "might" is just bad engineering.
Also, even if I do need async in a certain place, doesn't mean I need to endure the limitations and complexity of async Rust everywhere in my codebase. I can just spawn a single executor and pass around messages over channels to do what requires async in async runtime, and what doesn't in normal and simpler (and better) blocking IO Rust.
You need async IO? Great. I also need it sometimes. But that doesn't explain the fact that every single thing in Rust ecosystem nowadays is async-only, or at best blocking wrapper over async-only. Because "async is web-scale, and blocking is not web-scale".
Edit: Also the "just use smol" comically misses the problem. Yeah, smol might be simpler to use than tokio (it is, I like it better personally), but most stuff is based on tokio. It's an uphill battle for the same reasons using blocking IO Rust is becoming an uphill battle. Only thing better than using async when you don't want to is having to use 3 flavors (executors) of async, when you didn't want to use any in the first place.
Everything would be perfect and no one would complain about async all the time if the community defaulted to blocking, interoperable Rust, and then projects would pull in async in that few places that do actually need async. But nobody wants to write a library that isn't "web-scale" anymore, so tough luck.
IO is not a part of the async runtime contract (I don't know if this is good or bad), and Tokio & futures famously have different `Async{Read,Write}` traits. I once had to do this [0] to adapt between them.
This means that any crate that uses IO will be bound to a limited number of Runtimes. Everything being Tokio-only is pretty bad (though Tokio itself is great), but here we are...
In some sense it is. Async is a glorified future, and future is a glorified thread management, and threads are a way to facilitate asynchronous execution. You can also create a threadless runtime, but then you are relying on OS threads (e.g. I/O or XHR), otherwise you are simply combining function calls (for which we already have language syntax).
> But nobody wants to write a library that isn't "web-scale" anymore, so tough luck.
It's more like "I want to be able to put timeouts in my code". 99% of why I want async is so that if something takes too long I can just stop that. That is incredibly hard to do without async.
Not sure about Rust, but in other languages that don't have async: create a queue, spawn a thread with your task, thread with sleep and wait for a message from any of those two. Kill the still running thread when you get the message. Can't say it's incredibly hard (unless it's Javascript or you work in a single-threaded model in general).
> Kill the still running thread when you get the message
This is extremely difficult. I mentioned elsewhere that the only way to kill a thread is through the pthread_cancel API, which is very dangerous.
Languages with larger runtimes can get around this because they have all sorts of things like global locks or implicit cooperative yielding. So they don't ever have to "kill" a thread, they actually just join it when it yields.
As I understand when you are blocked on I/O and sends a signal to the waiting thread, that system call will simply be released and return an error. Ruby (Java etc.) does make it simple because of GC, so I don't need to worry about file descriptor leaks etc. But talking about Rust, shouldn't it be a part of a thread management? Basically if an error happens during normal blocking system call, it goes through the same sequence, no? E.g. you have to release any thread-local allocations no matter by each way system call was terminated. Rust threads are supposed to be memory safe, not sure about file descriptors. I don't quite understand what you mean by "yielding" though.
Genuinely curious: looks to me like you can set timeout on sockets (TcpStream) with std. Does that not work? What other kinds of timeouts do you typically need?
Relying on the socket APIs is a bit painful. For one thing, what if that socket is shared across tasks? What if I want to do per-request timeouts? What if I don't want to expose the socket APIs to callers?
What if my work isn't related to socket timeouts? For example, downloading a multi-part file? I may want a very long socket timeout but still have a distinct timeout for the individual chunked operations. It might take 30 seconds to grab an entire file but if one chunk takes >3 seconds I may want to time out.
What if my work involves no IO at all?
The ability to cancel work in progress is extremely important to me.
> For one thing, what if that socket is shared across tasks?
Yeah that’s an issue. In Go they sync it (thread safe), which in Rust would translate to interior mutability.
> What if I want to do per-request timeouts?
Ah you’re right in http2 there can be multiple concurrent reqs per conn. Go still allows request based timeouts, but I wonder if that’s possible with the limited primitives in std. It’s also true that this is a case where the inner conn should not be exposed.
> I may want a very long socket timeout but still have a distinct timeout for the individual chunked operations.
Right! That’s typically done by extending the deadline for every chunk. Ie the user/caller needs a way to set timeouts.
> The ability to cancel work in progress is extremely important to me.
Yes for sure. I was just curious. Btw which libs are you referring to for network requests? I’d like to see their APIs.
> I also need it sometimes. But that doesn't explain the fact that every single thing in Rust ecosystem nowadays is async-only, or at best blocking wrapper over async-only.
I guess one thing going for it is that then Rust is a lot more like Node, which could help increase its popularity. I wonder if Rust at some point in its history became self-serving (bigger == better).
>Except, this isn’t a problem with Rust’s async, it’s a problem with tokio. tokio uses a 'static, threaded runtime that has its benefits but requires its futures to be Send and 'static.
It's not a problem with tokio either. The author's point is specifically about the multi-threaded tokio runtime that allows tasks to be moved between worker threads, which is why it requires the tasks to be Send + 'static. Alternatively you can either a) create a single-threaded tokio runtime instead which will remove the need for tasks to be Send, or b) use a LocalSet within the current worker that will scope all tasks to that LocalSet's lifetime so they will not need to be Send or 'static.
If you go the single-threaded tokio runtime route, that doesn't mean you're limited to one worker total. You can create your own pseudo-multi-threaded tokio runtime by creating multiple OS threads and running one single-threaded tokio runtime on each. This will be similar to the real multi-threaded tokio runtime except it doesn't support moving tasks between workers, which means it won't require the tasks to be Send. This is also what the author's smol example does. But note that allowing tasks to migrate between workers prevents hotspots, so there are pros and cons to both approaches.
Actix-web uses the single threaded Tokio runtime per physical core. This architecture is harder to design for than multi threaded async Tokio. Performance gains aren't worth the effort.
We use the same design for a product at $dayjob and have had no difficulty in designing for it. There is a lot of benefit from being able to use Rc and RefCell instead of having to go for Arc and Mutex. (Of course it's best if it can be written to not have any RefCell / Mutex at all.)
I find async is so much fun in Python and meshes with the other things you can do with generators but that is because I have the reference collector cleaning up behind me.
Looking back with like 30 years of hindsight it seems to me that Java’s greatest contribution to software reuse was efficient garbage collection; memory allocation is a global property of an application that can’t efficiently be localized as you might want a library to use a buffer it got from the client or vice versa and fighting with the borrow checker all the time to do that is just saying “i choose to not be able to develop applications above a certain level of complexity.”
The problem with garbage collection is that it doesn't work for other kinds of resources than memory, so basically every garbage collected runtime ends up with an awkward and kinda-broken version of RAII anyway (Closeable, defer, using/try-with-resources, context managers, etc).
Static lifetimes are also a large part of the rest of Rust's safety features (like statically enforced thread-safety).
A usable Rust-without-lifetimes would end up looking a lot more like Haskell than Go.
> basically every garbage collected runtime ends up with an awkward and kinda-broken version of RAII anyway (Closeable, defer, using/try-with-resources, context managers, etc).
RAII works only for the simplest case: when your cleanup takes no parameters, when the cleanup doesn't perform async operations, etc. Rust has RAII but it's unusable in async because the drop method isn't itself async (and thus may block the whole thread if it does I/O)
In my experience async drop is a nice to have, not a must. Futures::block_on is good enough for the happy path in low stakes scenarios.
When dealing with async operations they tend to end up at a network boundary and thus the service enters distributed system land.
Now the async drop also has to handle the server crashing before the drop happens and at any time when it happens. Keeping that in mind trying to actually drop something becomes quite meaningless since you need to handle all other cases either way.
I agree but does RAII necessarily imply parameter-free destruction?
Personally I love Rust’s `fn foo(self, …)`, which is just like a regular method but consumes the value.
Deallocate by default is fine, but sometimes you need to run specific destructors (linear type style). I’ve long wished for an opt-out from implicit drop semantics for resource/handle types.
You can (kind of) emulate linear types by making `drop()` unlinkable[0]. Of course, I wouldn't recommend doing this since the error messages are awful and give no explanation at all of where the actual problem is...
People miss that there is a quantitative aspect to these things as well as a qualitative aspect. That is, many programs allocate millions of pieces of memory a second and for a wide range of different purposes whereas there might be a limited number of other resources of an even more limited set of types. Any change in the code probably has some affect on memory allocation, but many changes won't have any effect on allocation of higher-level resources.
Thus the complexity of handling memory is greater than that of other resources and the consequences of getting it not 100% right are frequently worse.
I quite like context managers and try-with-resources style constructs. They make lifetimes explicit in the code in a fairly intuitive way. You can get yourself turned around if you deeply nest them, etc but there are usually ways to avoid those traps.
You make an interesting point. Has any language introduced a generic-resource-collector? You're not supposed to use deconstructors to clean up resources because you're left to the whims of the GC which is only concerned about memory.
Has anyone build a collector that tracks multiple types of resources an object might consume? It seems possible.
Erlang is probably the closest. The word you want to search for is "port". If it doesn't seem like it at first, keep reading. It's a very idiosyncratic take on the topic of you view it from this perspective because it isn't exactly their focus. But it does have a mechanism for collecting files, sockets, open pipes to other programs, and a number of other things. Not fully generic, though.
Python handles all kinds of stuff with garbage collection.
The problem is that things like closing a socket are not just generic resources, a lot of the time nonmemory stuff has to be closed at a certain point in the program, for correctness, and you can't just let GC get to it whenever.
I don’t think this is true. Context managers call special magic “dunder” methods on the instance (I don’t remember the specific ones), and I’m pretty sure those don’t get called during regular garbage collection of those instances. It’s been a few years since I was regularly writing python, so I might be wrong, but I don’t believe that context manager friendly instances are the same as Rust’s Drop trait, and I don’t think their cleanup code gets called during GC.
Python is a fun case of "all of the above" (or rather, a layering of styles once it turns out a previous one isn't workable).
Originally, they used pure reference counting GC, with finalizers used to clean up when freed. This was "fine", since RC is deterministic. Everything is freed when the last reference is deleted, nice and simple.
But reference counting can't detect reference cycles, so eventually they added a secondary tracing garbage collector to handle them. But tracing GC isn't deterministic anymore, so this also meant a shift to manual resource management.
That turned out to be embarrassing enough that context managers were eventually introduced to paper over it. But all four mechanisms still exist and "work" in the language today.
Are you saying that a finalizer is guaranteed to run when the last reference is deleted? So you could actually rely on them to handle the resources, as long as you are careful not to use reference cycles?
In CPython 2.7, yes. In CPython in general, I believe it's currently still the case, but I don't think it's guaranteed for future versions.
For Python in general, no. For example, as far as I know Jython reuses the JVM's GC (and its unreliable finalizers with it).
It's also easy to introduce accidental cycles. For one, a traceback includes a reference to every frame on the call stack, so storing that somewhere on the stack would create an unintentional cycle!
The tracebacks were a lot of what made me cut back on using weakrefs and trying to make things manage their resources automatically.
Now I use close() methods for anything that needs to be closed. If I mess up and there's some obscure bug, hopefully GC will fix it, but it seems too brittle and easy to make mistakes with to rely on.
I agree that memory management can't be solved locally. The situation in C++, where every library or API you use has a different cleanup convention, that you need to carefully read about in the documentation to even properly review a pull request, is proof of that.
I disagree that this criticism applies to Rust. For 99% of the cases, the idiomatic combination of borrow checking, Box and Arc gets back to a unified, global, compiler-enforced convention. I agree that there's a non-trivial initial skill hurdle, one that I also struggled with, but you only have to climb that once. I don't see that there's a limit to program complexity with these mechanisms.
>The situation in C++, where every library or API you use has a different cleanup convention, that you need to carefully read about in the documentation to even properly review a pull request, is proof of that.
Lol wut. The C++ resource management paradigm is RAII. If you write a library that doesn't use RAII, it's a bad library. Not a fault of the language.
There’s a lot of C++ code out there and a lot that interfaces with C.
RAII is one method of cleanup but it doesn’t work in all situations. One that comes to mind is detecting errors in cleanup and passing them to the caller.
So it’s not right to call every library that doesn’t use RAII “bad.” There are other constraints, as well. Part of the strength of C++ is to give you a choice of paradigms.
Either you write code with good performance, which means that functions do take references and pointers sometimes, in which case you do have all of the usual lifetime issues. This is the proper way to use C++, and it's perfectly workable, but it's by no means automatic. That's the reality that my comment was referencing.
Or you live in a fantasy land where RAII solves everything, which leads to code where everything is copied all the time. I've lived in a codebase like this. It's the mindset that famously caused Chrome to allocate 25K individual strings for every key press: https://groups.google.com/a/chromium.org/g/chromium-dev/c/EU...
You're missing a bunch of very important stuff in that page you linked to. See what they listed as the culprits:
> strings being passed as char* (using c_str()) and then converted back to string
> Using a temporary set [...] only to call find on it to return true/false
> Not reserving space in a vector
c_str() isn't there for "good performance" to begin with; it's there for interfacing with C APIs. RAII or not, GC or not, you don't convert to/from C strings in C++ unless you have to.
The other stuff above have nothing to do with C++ or pointers, you'd get the same slowdowns in any language.
The language has come a long way since 2014. Notice what they said the solutions are:
I'm responding to a comment that claims all lifetime issues are solved by RAII.
My argument was that for efficient code, you need to pass references or pointers, which means you do need to care about lifetimes.
And your argument is that's not true because we now have std::string_view? You do realize that it's just a pointer and a length, right? And that this means you need to consider how long the string_view is valid etc., just as carefully as you would for any other pointer?
> I'm responding to a comment that claims all lifetime issues are solved by RAII.
I don't see anybody claiming this. The parent I see you initially replied to said "the C++ resource management paradigm is RAII", not "all lifetime issues are solved by RAII".
> My argument was that for efficient code, you need to pass references or pointers, which means you do need to care about lifetimes.
Of course you do. Nobody claimed you don't need to care about lifetimes. (Even in a GC'd language you still need to worry about not keeping objects alive for too long. See [1] for an example. It's just not a memory safety issue, is all.) The question was whether "every library or API you use" needs to have "a different cleanup convention" for performance reasons as you claimed, for which you cited the Chromium std::string incident as an example. What I was trying to point out was:
> that's not true because we now have std::string_view? You do realize that it's just a pointer and a length, right?
...because it's not merely a pointer and a length. It's both of those bundled into a single object (making it possible to drop them in place of a std::string much more easily), and a bunch of handy methods that obviate the ergonomic motivations for converting them back into std::string objects, hence preventing these issues. (Again, notice this isn't just me claiming this. The very link you yourself pointed to was pointing to StringPiece as the solution, not as the problem.)
So what you have left is just 0 conventions for cleanup, 1 convention for passing read-only views (string_view), 1 convention for passing read-write views (span), and 1 convention for passing ownership (the container). No need to deal with the myriads of old C-style conventions like "don't forget to call free()", "keep calling with a larger buffer", "free this with delete[]", or whatever was there over a decade ago.
> And that this means you need to consider how long the string_view is valid etc., just as carefully as you would for any other pointer?
Again, nobody claimed you don't have to worry about lifetimes.
I agree that a lot of that happens in the real world. I disagree that RAII is not used in the real world. I worked on a very large codespace for ATM client software and we used it pervasively, and the only memory leak we had in my time there was in a third-party library which ... required the careful reading of documentation you mentioned.
widely used though. not sure if that count for appreciation, but i think it's one of the highest forms.
it's not bad, not not great either. i miss proper sum types, and it really lament the fact that static things are nearly impossible to be mocked which prompts everyone to use DI for everything instead of static.
Java has sum types now with sealed interfaces and pattern matching. Records have detructoring out of the box, and I believe supporting it for general classes is in the works.
sealed interface Shape {}
record Square(int x) implements Shape {}
record Rectangle(int l, int w) implements Shape {}
record Circle(int r) implements Shape {}
double getArea(Shape s) {
// Exhaustively checks for all alternatives.
return switch (s) {
case Square(var x) -> x * x;
case Rectangle(var l, var w) -> l * w;
case Circle(var r) -> Math.PI * r * r;
}
}
Loved the article. Thanks for sharing. Maybe I (coming from Haskell and Elm) should not be bothered too much with the verbosity of Java/Kotlin "sum types" :)
The only Java thing I work with is ElasticSearch (and in the past other lucene based search tools like Solr). These can be resource hungry depending on what your indexing but they are also faster and more scalable than other tools I’d used before.
> I’ve written quite a few Rust projects where I expect it to only involve blocking primitives, only to find out that, actually, I’m starting to do a lot of things at once, guess I’d better use async.
In my experience (which, admittedly, is far less than the author, a developer of smol!) the answer to "I'm starting to do a lot of things at once" in Rust is usually to spin up a few worker threads and send messages between them to handle jobs, a la Ripgrep's beautiful implementation.
In a way, it seems like async Rust appears more often when you need to do io operations, and not so much when you just need to do work in parallel.
Of course, you surely can use async rust for work in parallel. But it's often easier to keep async out of it if you just need to split up some work across threads without bringing an entire async executor runtime into the mix.
I don't think async/await was poorly implemented in Rust - in fact, I think it avoids a lot of problems and pitfalls that could have happened. The complications arise because async/await is, kind of, ideologically antithetical to Rust's other goal of memory safety and single-writer. Rust really wants to have its cake (compile-time memory safety) and eat it too (async/await). And while you can criticize it, you have to admit they did a pretty good job given the circumstances.
That's pretty simple. The primary goal of every software engineer is (or at least should be) ... no, not to learn a new cool technology, but to get the shit done. There are cases where async might be beneficial, but those cases are few and far in between. In all other cases a simple thread model, or even a single thread works just fine without incurring extra mental overhead. As professionals we need to think not only if some technology is fun, but how much it actually costs to our employer and about those who are going to maintain our "cool" code when we leave for better pastures. I know, I know, I sound like a grandpa (and I actually am).
You can cancel socket operations using signals. You can eg have one or more background threads running timers which will interrupt the blocking IO if it doesn’t return in a timely manner. A lot of very important frameworks and services that are used in billions of transactions per day use this model.
Of course you can. It does mean that you need cooperation between the child and parent thread (to set up the signal handler so that resources are cleaned up) though. That's easy in a framework, kind of a pain in the ass if you're just trying to get some opaque client you were passed to do something in <10 seconds.
And that's just for IO. I mentioned elsewhere that you may want to cancel pure compute work.
You can see my point, I assume, that when your userspace program can cancel tasks natively it's much easier to work with?
Can you cancel a tight computing loop (i.e. without system calls and without yielding of any sort) with async? I wonder how? Also if you can inject a cleanup code in your async task what prevents you from doing it with threads? Such things existed long before async/await and system calls didn't change for async/await. Also, what's the difference between "framework" and async/await runtime, isn't the latter a kind of a framework?
Without any yielding? Seems hard. You could park the thread idk.
> what's the difference between "framework" and async/await runtime,
Sure, in that in both cases you have the threads managed for you. But there's a difference between spawning a raw pthread, which will have no signal handlers/ cleanup hooks, and one managed by a framework where it can add all of those things and more.
Interesting, in the Java world Thread.stop is deprecated too: https://docs.oracle.com/javase/7/docs/technotes/guides/concu... Which means there is no good way to actually stop a thread involuntary. Of course in most simple apps it's not a big deal, but I would not do it in long-running apps.
OTOH in Rust async model is based on polling. Which means that poll may never block, but instead has to set a wake callback if no data is available. So there is no way to interrupt a rogue task and all async functions should rely on callbacks to wake them (welcome to Windows 3.1, only inside out!). Thread model is much more lax in this sense, e.g. even though my web server (akka-http) is based on futures, nothing prevents me from blocking inside my future, in most cases I can get away with it. As I understand it's not possible in Rust async model, I can only use non-blocking async functions inside async function. So in reality you don't interrupt or clean up anything in Rust when a timeout happens, you simply abandon execution (i.e. stop polling). I wonder what happens with resources if there were allocated.
> As I understand it's not possible in Rust async model,
You can block, you're just going to block all other futures that are executed on that same underlying thread. But all sorts of things block, for loops block.
This is the same as Java, I believe. Akka also has special actors called IO Workers that are designed for blocking work - Rust has the same thing with `spawn_blocking`, which will place the work onto a dedicated threadpool.
> So in reality you don't interrupt or clean up anything in Rust when a timeout happens
You don't interrupt, you are yielded to. It's cooperative.
> I wonder what happens with resources if there were allocated.
When a Future is dropped its state is dropped as well, so they are all freed.
If you block your timer goes out the window, right? Because the poll will never get there until the blocking call is done. So yeah, you can block, but it will disrupt the whole chain, including tasks above yours up to await. Similar to Erlang VM where the language itself yields (e.g. there are no loops and every recursive call is effectively a yield), but if you add a C module and are careless enough to block, the whole EVM blocks. So no, if you want to use async you shouldn't block. For loops? Nope, not if they take long time for the same reason, you may want to break them down to smaller chunks ("long" depends on other tasks and expected latency).
Having said that, Erlang exists and doing well, so async is as good as any model designed for special cases. But this discussion basically answers the question
> Why don’t people like async?
Because not everybody (which means a majority of developers) needs this complexity. And the upward poisoning means that I can't block in my function if my web server is based on async, which affects everybody who is using it.
> If you block your timer goes out the window, right?
This is the case in every language.
> So no, if you want to use async you shouldn't block.
Everything blocks. The dosage makes the poison.
> For loops? Nope, not if they take long time for the same reason
You would want to add a yield in your loop, yes. Async loops `while let Some(msg) = stream.next().await` will work well for this.
> And the upward poisoning means that I can't block in my function if my web server is based on async, which affects everybody who is using it.
To be clear, you can definitely block as much as you want in those frameworks, you just need to understand that you'll block the entire thread that's running your various futures. That's not that big of a deal, you'd have the exact same issue with a synchronous framework. Blocking in an OS thread still blocks all work on that thread, of course.
The last comment is actually pretty interesting and spot on. In the Java/JDK world - which you can assume as a „framework“ - you can cancel blocking IO via the Thread.interrupt() mechanism. And that works because it’s deeply integrated into the framework, similar like async Rust runtimes provide support for cancellation.
> Show me how to cancel a network requests using only threads, with no access to the underlying socket APIs?
It’s been a long time since I did this in Rust. But why do you not have access to the sockets or at least a set_timeout method? Is it a higher level lib that omits such crucial features?
In Go, the super common net.Conn interface has deadline methods. Not everyone knows their importance but generally you have something like it piped through to the higher layers.
EDIT: Oh I see you replied to my other comment. Please disregard.
Total rust newb here, but does that need the full async story, or is it a limitation of an API somewhere? From the point of view of the code using the request's response could you use a channel with recv_timeout? Is the problem there that the thread with the socket connection is still going and there's no way to stop it?
The ability to cancel an operation without talking to the operating system requires that your program has yield points. That yielding is what allows another part of the program to take control and say "OK, I'm done with you now, no need to finish".
Yes, the problem is that your thread would continue to perform work even if you stopped waiting on it.
Maybe I don't understand the complexity, but in good old Ruby I can easily stop a thread if I don't need result anymore. No async needed and no yield points necessary. Doesn't it apply to Rust too?
I assume that Ruby does in fact have yield points in some form, such as a global lock. Killing a thread is only possible (for a pthread) via the `pthread_cancel` API. That API is very dangerous and is generally not something you'd ever want to use manually - the thread will not clean up any memory or other resources, any shared memory is left in a tricky state.
To gracefully shut a thread down you need yielding of some kind.
Most commercial code is running an almost entirely IO workload, acting as a gatekeeper to a database or processing user interactions - places where async shines.
Async isn't a lark, it's a workhorse. The goal is not to write sexy code, it's to achieve better utilization (which is to say, save money).
Depends on the nature of commercial code and if it has another level of parallelism (think of web servers and read my comment below). As for DB queries, here's the thing: most commercial code is using DB transactions and there is no way to run transaction across multiple connections, so you are either single-threaded and do things in sequence anyway (why use async then?), or you are multi-threaded and then forget about transactions. Besides that, even if you can get away with multiple transactions there are those pesky questions like "what to do with a partially failed state?". Not all transactions are idempotent, and not all are reversible, it's hard enough when you run them sequentially, and running them in parallel and dealing with a failure might be an absolute nightmare.
Most web applications (every one I've ever worked on) use connection pooling to run multiple transactions in parallel. I suppose you could think of that as a sort of network level parallelism, but it's not multithreading.
Connection pooling is of course not without it's hazards, scaling databases can be very difficult and almost all of the production incidents I've dealt with involve a database running out of a resource (often connections). But for your garden variety web app, it certainly isn't a dichotomy between serializing all concurrent updates or losing atomicity.
But async Python is a single threaded. I’d prefer async over multithreading in python nowadays. Otherwise code can be slow as piss, if it’s doing a lot of I/O. Then, async is almost table stakes for almost any level of reasonable performance (GIL and all).
Not exactly sure how async in Python works, but if its runtime is non-preemptive and single-threaded (i.e. based on yield), then congratulations, you reinvented Windows 3.1! Those who are old enough to be "lucky" to use it, remember that the damn thing could hang the whole OS if your application was careless enough to block and not yield. Also "slow" is relative, if you create a thread to do DB query, thread creation is a way faster than any DB request, so not sure why it's slow. Never had problems with Ruby threads even though Ruby doesn't have a mechanism to create a thread pool (didn't have? it's been some time since I worked with Ruby). Java & Scala, OTOH are using thread pools, even multiple variations of them, so the thread startup time doesn't matter. In any case you are talking about I/O, in which case neither thread startup nor context switching matters.
Another reason to is that it lets you handle bursty input with bursty CPU usage. Sounds great, right? Round peg, round hole.
But nobody will sell you just a CPU cycle. They come in bundles of varying size.
I recently heard a successful argument that we should take the pod that's 99% unutilized and double its CPU capacity so it can be 99.9% unutilized, that way we don't get paged when the data size spikes.
When I proposed we flatten those spikes since they're only 100ms wide it was sort down because "implementing a queueing architecture" wasn't worth the developer time.
I suppose you could call it a queueing architecture. I'd call it a for loop.
Your answer boils down to: "I know this technique, I don't want to learn blub technique. My job is to get stuff done, not learn new techniques." In which case, good for you; enjoy your sync code (seriously), and please stop telling the rest of us that have learnt the new blub technique that we shouldn't use it.
Honestly, your dismissal of its value sounds very much like you don't know how to use it. The whole argument can be turned around and the same said about threads, which are not "simple" as you suggest if you don't already know how to use them. You might as well say "simple async".
If you carefully read my message, I said there are cases where async is beneficial. Most of the time I don't think even threads are necessary. E.g. the most common application nowadays (arguably) is a web server. Of course those who write web server itself may use whatever technology that fits, but for us mortals who simply want to receive request, query DB and respond with data, even threads have a very limited usage. Why? Because web servers are highly parallel, you try to make your request processing parallel and you starve another request (DB is a limited resource, and most web apps don't require computational power). So a simple sync processing works just fine -- no headaches, no mental overhead and you can focus on the business logic, that's what your employer values the most. The exception is when your company name is Twitter or X, whatever (which the most of web apps are not). Other cases? Depends, but the same approach applies: we usually have a bottleneck somewhere else, so you are trying to be smart and starves that. And introducing a sophisticated approach where it's not necessary you shift the focus from the business logic (see above).
Also parallel execution is never simple, there are multiple problems no matter what technology you use, be it async or threads. Meanwhile there are different threads too, you know, green, system etc. There is Erlang for example, which existed long before async was invented. Async is just the current hype, which always starts with "we solved this specific problem, let's do it everywhere!", then ... yeah, we did, but only for this special case, but then it creates tons of problem elsewhere, but we are not going to look there, and if you are looking there we will declare you simply not able learn our new shiny thing. Been there, seen that. Even had this mentality.
I'd argue that libraries forcing programs to include an async runtime upfront because there's a chance that it may someday grow to the extent you want a central executor (when they are absolutely ill-suited for audio programming, and probably not the case for Animats's metaverse client at https://news.ycombinator.com/item?id=37437676), imposes unnecessary dependency bloat on applications. And unless you use block_on(), async code pushes applications from a "concurrency library" model where apps block on mutexes/signals/channels as needed, to a "concurrency framework" model where code only runs under the runtime's callbacks (which is not the right choice for all apps).
The author starts by citing greenspun's tenth rule and goes on to elaborate on the argument that if you are going to have a half implementation of async anyway, why not just pull it in? Yet fails to interrogate the relationship between this argument and the cited "rule". If you should use async because you might need it in the future, shouldn't we all be writing in lisp?
If we presuppose that all software eventually develops an async, and we therefore should use async. Would it not stand to reason that greenspun's rule that all software contains a lisp would imply that we must also all use lisp?
The author said what you wrote in the first sentence, ie "use async if you are going to have a half implementation of async anyway". "Use async because you might need it in the future" is something you made up, not what the author said.
greenspun's tenth rule is about the inevitability of the half baked implementation of lisp. By evoking the sentiment of the rule the author is implicitly making the argument that all "sufficiently complicated programs" will eventually contain a half baked implementation of async.
The implicit argument doesn't stand alone though. The author goes on to write:
> It happens like this: programs are naturally complicated. Even the simple, Unix-esque atomic programs can’t help but do two or three things at once. Okay, now you set it up so, instead of waiting on read or accept or whatnot, you register your file descriptors into poll and wait on that, then switching on the result of poll to figure out what you actually want to do.
The implication is clear. Even simple programs will eventually require async, and should therefore just use it right now. unix-esque in this paragraph is supposed to evoke ls or cat. Is your program really going to be simpler than cat? No? Then you apparently need async.
>The implication is clear. Even simple programs will eventually require async, and should therefore just use it right now.
There's no implication. Read what you quoted instead of digging for quick jabs. "Even the simple, Unix-esque atomic programs can’t help but do two or three things at once. Okay, now you set it up so, instead of waiting on read or accept or whatnot..."
>unix-esque in this paragraph is supposed to evoke ls or cat. Is your program really going to be simpler than cat? No? Then you apparently need async.
The rule isn't really about lisp, it's about the kinds of functions and structures you find in the standard library of a typical programming language, such as strings and arrays and file IO and so on. Rust already has those things so your argument doesn't really apply.
I love async in Python and JS. I used to be one of those "Threads aren't that hard, just use threads" people, but that was back when the trend was doing "async" with layers of nested callbacks like as if this was LISP or something where people just accept deep nesting.
Now we have async/await and I'm always happy to see it.
> Even the simple, Unix-esque atomic programs can’t help but do two or three things at once. Okay, now you set it up so, instead of waiting on read or accept or whatnot, you register your file descriptors into poll and wait on that, then switching on the result of poll to figure out what you actually want to do.
> Eventually, two or three sockets becomes a hundred, or even an unlimited amount. Guess it’s time to bring in epoll! Or, if you want to be cross-platform, it’s now time to write a wrapper around that, kqueue and, if you’re brave, IOCP.
This feels like a straw man. Nobody is saying "don't use async; use epoll!". The alternative to async is traditional OS threads. This option is weirdly not mentioned in the article at all.
And yes they have a reputation for being very hard - and they can be - but Rust makes traditional multithreading MUCH easier than in C++. And I would argue that Rust's async is equally hard.
Rust makes traditional threading way easier than other languages, and traditional async way harder than other languages, enough that threads are arguably simpler.
True but you can still do that using traditional threads using cancellation tokens.
In some ways it's worse because you have to explicitly add them, and I have yet to see any Rust APIs that actually use them (though there is a `cancellation` crate so at least some must be).
In other ways it's better because it gives you control and explicit visibility over the cancellation points.
Do you have a source on these cancellation tokens for threads?
The cancellation crate hasn't been touched since 2016 and requires the running thread have a mechanism to be woken up. If you're in the middle of a read, you won't observe a wakeup unless you use an async-io function that can be timed out or interrupted.
This is no better than async/await. And await is just as obvious of the cancellation points.
That being said, there are also numerous crates for async rust cancellation tokens that can polled in parallel with a read such that you can observed the cancel instantly and switch to a cleanup process instead of immediately cancelling everything
You wait on a shared channel so both read and sleep threads queue a message when ready (whichever comes first). Not sure about channels, but in other languages it would be a concurrent queue.
The author is a maintainer of smol, which I think is a far superior runtime to tokio for numerous reasons including structured concurrency, performance, size, and an ownership parameter that reduces the need for Arc<> all over the place by letting you scope on the runtime or task. The whole thing is just tighter and better thought out.
Yet tokio is the de facto standard and everything links against it. It’s really annoying. Rust should have either put a runtime in the standard library or made it a lot easier to be runtime neutral.
Lots of comments and arguments about async being big or complex, and it's really not, it's pulling in the runtimes that's big and complex, and I think Rust really failed by forcing libraries to explicitly choose a runtime. As a library developer you're then put in the position of not using async, or fragmenting yourself to just the subset of users or other libraries on your runtime.
Biggest issue I have with async is the lack of native async traits and the lack of support for async closures. You can work around the traits issue but the closure issue you can’t. I’ve spent hours trying to work around closures that wrap async code.
do any of the async libraries for rust have good visualization tools for inspecting the implicit state machine that is constructed via this type of concurrency primitive?
The state machine transformation is not specific to any async libraries. The compiler is the one that desugars async fns / blocks to state machines. AFAIK there is nothing other than dumping the HIR / MIR from rustc to inspect it. But even without that the transformation is pretty straightforward to do mentally.
The first transformation is that every async block / fn compiles to a generator where `future.await` is essentially replaced by `loop { match future.poll() { Ready(value) => break value, Pending => yield } }`. ie either polling the inner future will resolve immediately, or it will return Pending and yield the generator, and the next time the generator is resumed it will go back to the start of the loop to poll the future again.
The second transformation is that every generator compiles to essentially an enum. Every variant of the enum represents one region of code between two `yield`s, and the data of that variant is all the local variables that in the scope of that region.
Am I the only one that after reading opening sentences like
"There is a common sentiment I’ve seen over and over in the Rust community that I think is ignorant at best and harmful at worst."
just refuses to read the rest? If you are actually trying to make a point to people that think differently than you, why antagonize them by telling them they don't know what they are talking about?
> tokio uses a 'static, threaded runtime that has its benefits but requires its futures to be Send and 'static.
This is only partly true -- if you want to `spawn` a task on another thread then yes it has to be Send and 'static. But if you use `spawn_local`, it spawns on the same thread, and it doesn't have to be Send (still has to be 'static).
For the thing I’m working on, I have an infinite number of little tasks with potentially shared smaller subtasks.
How could I unleash all the processors on my computer on this workload and allow them to correctly avoid repeated calculation of results of shared subtasks?
For example, I’m using an outbox: im::OrdMap<String, Array2<_>> and a situation might arise where one task could avoid repeating work on a subtask because that’s already in progress elsewhere by waiting for the key/value pair (so that process could do something else)
Would it be worth going to async for that?
How could a worker function know if some key in the outbox was already being calculated and it could work on something else?
How would you share an outbox like that across a bunch of rayon processes communicating with async?
(I’ll read smol docs and try to figure it out but this article made a lot of sense, thank you)
When does async/await ever make your program harder to maintain? Maybe to people who don't already know it, but almost all the big languages have async, it would be hard for a programmer to get away with not learning it, at least if they're a python of JS programmer.
It adds complexity, but it's at the level where you don't have to think about it. If you're doing something advanced enough to where async is a leaky abstraction, you're probably doing something big enough to where you would want the advantages it offers.
If you're doing something simple, async is just a black box primitive that is pretty easy to use.
This is not true for Rust. Await in rust builds a larger state machine from the former. It does no implicit thread or task spawns (unless the future you're awaiting does them explicitly).
Furthermore, async rust can be run single threaded
Then you block the runtime? _aha you got me, threads and pre-emptive concurrency is better_.
This is where you have a reasonable trade off. I have accepted that async gives me more control over my code. For that I have to accept that blocking can slow down the app. After running async rust in production for over 2 years now I've not seen any blocking tasks block the executor. Maybe I'm just good but my experience is that my colleagues who come from C# generally don't make these mistakes either
FastAPI docs, case when you don't create an async route
> When you declare a path operation function with normal def instead of async def, it is run in an external threadpool that is then awaited, instead of being called directly (as it would block the server).
NB: In Python >= 3.9 the idiomatic way to do this is to_thread(), not familiar with these ASGI functions but I would guess they're a polyfill and/or predate 3.9.
They are not polyfills. Multiple scheduling modes are provided for libraries that are not thread safe (it's a total mess and I avoid these wrappers like the plague)
That is an implementation detail on where you put the code that is blocking or running concurrently from the main code. An executor could use a separate OS thread, or the application could itself schedule application levels threads onto a number of OS threads.
When writing a Future that will block for 5 seconds you will need to find somewhere to that you can put the code to block for 5 seconds. You don't technically need to even use an executor here.
I’ve worked deeply in an async rust codebase at a FAANG company. The vast majority chooses a dialect of async Rust which involves arcs, mutices, boxing etc everywhere, not to mention the giant dep tree of crates to do even menial things. The ones who try to use proper lifetimes etc are haunted by the compiler and give up after enough suffering.
Async was an extremely impressive demo that got partially accepted before knowing the implications. The remaining 10% turned out to be orders of magnitude more complex. (If you disagree, try to explain pin projection in simple terms.) The damage to the ecosystem from fragmentation is massive.
Look, maybe it was correct to skip green threads. But the layer of abstraction for async is too invasive. It would have been better to create a “runtime backend” contract - default would be the same as sync rust today (ie syscalls, threads, atomic ops etc – I mean it’s already half way there except it’s a bunch of conditional compilation for different targets). Then, alternative runtimes could have been built independently and plugged in without changing a line of code, it’d be all behind the scenes. We could have simple single-threaded concurrent runtimes for embedded and maybe wasm. Work stealing runtimes for web servers, and so on.
I’m not saying it would be easy or solve all use-cases on a short time scale with this approach. But I do believe it would have been possible, and better for both the runtime geeks and much better for the average user.