Without a GIL it would be incredibly difficult, would it not? AFAIK C extensions to MRI are not thread safe.
This could, of course, expose a different C API that allows for thread safe extensions, but I think the original comment is complaining about lack of compatibility with all of the important gems that depend on the existing MRI C API.
This has always been an issue that held back JRuby.
Hi, I'm the author of Artichoke. I don't have a good answer for whether or not implementing the MRI API would be difficult. I _suspected_ it would be, which is why I listed it as a non-goal. There is a lot of other work to do before we get to the point of C API compatibility.
You might want to clarify in the readme that the red X means non-goal. Until I read this comment I thought it was a goal that had not yet been implemented.
in the 14 hours since I wrote the comment you replied to, I moved an MRI C API from non-goal to goal [0]. :) The red X means the goal is not yet achieved.
Could one add, effectively, a RWLock GIL - one where any number of Ruby interpreter threads can run at once (since they know how to not step on each other's toes), but any C code which tried to "grab the GIL" would run exclusively w.r.t. interpreter threads (and other C code)?
Something similar to Racket places or V8 isolates can be implemented and keep compatibility with C extensions expecting GIL. It just needs somebody to actually do the work of swapping out hundreds of global variable accesses in MRI using something like CIL.
Do you mean every thread would have its own place/isolate for executing C extensions?
I think one difficulty there is some global state in C extensions might expect to be truly process-global.
Also, how would you isolate global variables in the C extension? If the C extension is a dynamic library, it's typically only loaded once per process.
Maybe something like dlmopen() to load multiple copies of a native library would help, but it's not portable.
Yeah, this is a real problem. The PG gem, for example, uses some global variables.
IMHO the only way to do it is to add incremental parallelism which leaves the GIL in place. Racket has already shown a solid path here.
Guilds would have a major performance problem: can't allocate objects without GIL. It's also a tricky mental model and requires invasive changes to existing Ruby code to handle frozen objects.
Places don't share a heap so they don't need the GIL to allocate objects and have independent GC rather than global GC. It's a model which fits better with existing Ruby code. The GVL can be relaxed while executing Ruby and grabbed by native methods.
> Guilds would have a major performance problem: can't allocate objects without GIL.
Why not? It'd be possible to have TLABs, isn't it? But yes, GC would still be for all Guilds at once.
Racket places don't allow to share objects, only arrays of primitive types, which seems very restrictive. And Racket futures are even more restricted.
I'm not sure about Guilds since it's not there yet, but it already sounds closer to Ruby multithreading and more flexible to me from an usage point of view.
TLABs are definitely possible but it's increasing the implementation complexity.
Places are deliberately shared-nothing to reduce the implementation complexity and provide a simple model to the user. Process oriented code can be easily ported.
Porting something like a Sidekiq based worker using ActiveRecord to work inside a Guild where all access to shared objects needs to be frozen would be a nightmare.
With Racket Places you can share more complex types but it needs to be done via FFI. It doesn't prevent you from implementing a sharable connection pool or anything.
The proposed API, MRB_API, is used in mruby. With enough popular support for that C API, it could force mainline ruby to modernize it's C API to that one.
Assuming MRB_API thread-safe-able. mruby is not thread safe at all, being a single-threaded script runner.
My expectation is that MRB_API can be implemented in a thread-safe way. The API manipulates the interpreter state (in the case of mruby, an `mrb_state`). The interpreter state itself and the implementation of the API can be thread safe or they can not be.
I don't think mruby even assumes the presence of threads on target systems, so it is completely reasonable that the implementation is not thread safe.
It also lacks compatibility with a lot of important C gems (not only or even predominantly because of the lack of a GIL, of course), which I’m pointing out has always impacted its adoption.
That’s Truffle, not JRuby. There was a plan several years ago to move Sulong support into JRuby based on trialing it in Truffle; it appears to have made no progress since 2016.
My personal experience with Ruby is extremely shallow, but a casual glance at the search results wrt the GIL in Ruby seems to say it actually does have one. I could be wrong, as I'm not an experienced Ruby developer, however.
