IMO the big win for Elixir/Nx/Bumblebee/etc is that you can do batched distributed inference out of the box without deploying anything separate to your app or hitting an API. Massive complexity reduction and you can more easily scale up or down. https://hexdocs.pm/nx/Nx.Serving.html#content
I've been really curious about BEAM languages but never made the leap. How well does it manage heterogeneous compute? I'm used to other languages making me define what happens on CPU vs GPU and defining cross-machine talk around those kinds of considerations.
What parts of that does elixir (and company) allow me to not write? Is there a good balance between abstractions when it comes to still maybe wanting control over what goes where (heteregeneity)?
Super curious and kinda looking for an excuse here :)
The BEAM is pretty high level, and it's REALLY good at managing distributed compute at the thread or device level.
If you have a parallelizeable workflow, it's very easy to make it (properly!) parallel locally, where by "properly" I mean having supervision trees, sane restart behavior, etc.
And once you have that you can extend that parallelism to different nodes in a network (with the same sanity around supervision and discovery) basically for free. Like, one-line-of-code for free.
Nonetheless, it's all message-passing, and so pretty high level. AFAIK it's not designed for parallelizing compute at GPU scale.
That being said, if you have multiple GPUs and multiple machines that have to coordinate between them, Elixir/Erlang is pretty much perfect.
If you have a multistage workflow with concurrent and possibly heterogenous requests that hit all the time you can very easily batch and distribute the workflow among compute resources in "the most natural way possible" easily, without having to resort to grace periods, etc. which introduce latency. I think that would be much harder to accomplish with java.
You might want to look at the java aparapi project
Aparapi allows Java developers to take advantage of the compute power of GPU and APU devices by executing data parallel code fragments on the GPU rather than being confined to the local CPU. It does this by converting Java bytecode to OpenCL at runtime and executing on the GPU, if for any reason Aparapi can't execute on the GPU it will execute in a Java thread pool.
From a "marketing strategy" point of view, I wonder what the thinking is in investing in this stuff so heavily when Python seems to be kind of the go-to? Will they be able to create a "good enough" environment to do that kind of work with Elixir? Is it just someone or a company scratching their own itch?
This is a genuine question - I don't know much about "AI stuff", but do know something about the economics of programming languages and I'm "intellectually curious" about what is driving this and what the goals are, rather than critical of Elixir. I love working with BEAM and miss it.
One factor may be that a few years back the language creator (José Valim, also the author of this article) announced that the language is basically "completed", and that they would shift focus to other things like developer tooling and other projects outside of the language itself.
José is quite prolific, so I think it's natural that he moves on to things like this. It's hard to know what reception will be like until you build it.
From a strictly "marketing" point of view, if you want to grow the language and ecosystem, it seems the successful move is to stake out a place where you're likely to win.
I think often this happens more or less by accident rather than conscious design - but think of something like PHP which made it really easy to whip up quick web pages, or Rails which drive Ruby adoption for a better, more structured, but still dynamic and quick web programming experience.
And I suppose part of those happy accidents are people just hacking on something they think is cool, so I wouldn't stress too much about the "marketing aspect". I'm just curious what drove it.
I have always considered helping the community grow into a diverse ecosystem to be my main responsibility (the Python community being a great example here).
This particular effort started because some people got together and realized that we could do it! Do it in a way that felt part of Elixir and not just a bunch of bindings to C libraries.
We honestly never had the expectation that we had to beat Python (otherwise we would simply not have started). Early on, we were not even sure if we could be better at one single thing. However, 3 years later, we do have features that would be quite hard or impossible to implement in Python. For example:
* Nx Serving - https://hexdocs.pm/nx/Nx.Serving.html - allows you to serve machine learning models, across nodes and GPUs, with concurrency, batching, and partitioning, and it has zero dependencies
* Livebook - https://livebook.dev - brings truly reproducible workflows (hard to achieve in Python due to mutability), smart cells, and other fresh ideas
* A more cohesive ecosystem - Nx, Scholar, Explorer, etc all play together, zero-copy and all, because they are the only players in town
Of course, there are also things that Python can do, that we cannot. The most obvious ones being:
* In Python, integration with C code is easier, and that matters a lot in this space. Python also allows C to call Python, and that's just not possible in the Erlang VM
* Huge ecosystem, everything happens in Python first
At the end of the day, what drives me is that the Erlang VM offers a unique set of features, and combining them with different problems have historically lead to interesting and elegant solutions. Which drives more people to join, experiment, run in production, and create new things.
Thanks for the great answer! And thanks for your work with Elixir. As a long-time Erlang user, I was impressed at how it wasn't just a slightly nicer syntax, but included genuine innovations and improvements.
My guess is that Jose and the core team are both personally interested in the big wave of ML stuff we've been experiencing recently and also want to demonstrate that Elixir is a viable platform for doing this work to teams which have adopted Elixir and are interested in ML but don't want to add a bunch of Python into their codebase.
I would also guess that Elixir led to some crazy things they didn’t even imagine possible with web, like phoenixs live view [1]. Even if they don’t have explicit ideas of how it will impact ML, it’ll be really interesting to try.
Correct. When I did the presentation, I did mention we attempted to type Elixir and failed. Therefore, the idea of saying the language was "complete" was precisely to signal the community that most of the work would happen in the ecosystem and not by directly changing the language. Especially because:
* The language was designed to extensible - see how Numerical Elixir can compile Elixir to the GPU and it didn't require changes to the language (we improved the language based on some needs but it was never a blocker)
* A lot of the hard work that makes Elixir better every year is actually happening on the Erlang VM, which we mostly get to leverage by doing nothing (for example, Elixir "got" a JIT since I said we were complete and we had to change nothing)
I'd say the definition of "completed" still holds, because we haven't announced changes to the language yet, but if the work on the type system succeeds, then we will introduce a new typing annotation, which is definitely big enough to void the label after 7-8 years.
Strictly optional and without changing the language API, fwiw. So about as smooth/painless an experience as possible.
Its addition into the next minor verision (1.17) will bring warnings that address some of the most common footguns in the language, like comparing structs.
I actually think the BEAM is an ideal environment for machine learning, sharding things across machines. The only thing I’m not sure of is if PyTorch etc. are more optimised than XLA the backend Axon uses… would be good to see some performance comparisons of a big LLM running on both. For everything else I’d suggest Elixir was a better experience.
It's very serious. The BEAM had a problem in that it lacked solid number-crunching libraries, so some folks solved that, and when they did distributed ML-capabilities kind of just fell out as a neat bonus so some folks did that too.
So now it's integrated into the basic Livebook, just boot it and go to that example and you have a transcriber or whatever as boilerplate to play around with. Want something else from Huggingface? Just switch out a couple of strings referencing that model and all the rest is sorted for you, except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.
Python is awful for sharding, concurrency, distribution, that kind of thing. With the BEAM you can trivially cluster with an instance on a dedicated GPU-outfitted machine that runs the LLM models or what have you and there you have named processes that control process pools for running queries and they'll be immediately available to any BEAM that clusters with it. Fine, you'll do some VPN or something that requires a bit of experience with networking, but compared to building a robust, distributed system in Python it's easy mode.
I don't know what the goals are, but I perceive the Nx/Bumblebee/BEAM platform as obviously better than Python for building production systems. There might be advantages to Python when creating and training models, I'm not sure, but if you already have the models and need to serve more than one, and want the latency to be low so the characteristically slow response feels a little faster, and don't already have a big Kubernetes system for running many Python applications in a distributed manner, then this is for you and it'll be better than good enough until you've created a rather large success.
> except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.
We use Elixir for our primary application, with a fair amount of Python code to manage our ML pipelines. But we also need real-time inference and it's really convenient/performant to be able to just do that in-app. So I, for one, am very grateful for the work that's been done provide the level of tooling in Elixir. It has worked quite well for us.
I can't say I know much about AI stuff or BEAM. But my best guess is that elixir native ML should integrate well with OTP's distributed computing capabilities. As an outsider to the elixir ecosystem, I've seen glimpses of elixir ML here and there but no mention of attempting to bridge the python ML ecosystem into elixir.
To the author, I noticed a typo: a misspelling of "meachine" instead of "machine"
"""
These features bring Numerical Elixir and its ability to setup distributed model serving, over CPUs and GPUs, to traditional meachine learning algorithms, allowing developers and data practitioners to tackle a wider number of problems within the Elixir ecosystem.
"""
My experience with Elixir onboarding was meh. Spent hours trying to setup the LSP in VSCode and Neovim. Their pseudo-official LSP (elixir-ls) didn't work at all. I even made a post about it on Reddit, Github, and here. No one really knew what was going on.
Even with Haskell you have something like ghcup and you're good to go. Not to mention Rust's amazing Cargo and Go's tooling as well.
So far, Elixir has been even more challenging to just get up and running than Common Lisp!
By the way, the official Elixir website recommends using Homebrew to install it. But almost everyone in the Github issues and comments says ASDF is the way to go.
ASDF is easy to use once you've learned a few of the basic commands, and that way you'll have an easy time when new versions come out and want to check out new features. Like the built-in JSON parser, the gradual typing when it drops, things like that.
If you built something useful you might not want to upgrade it just to look at the new stuff, and if you built nothing and just drop into iex a couple of times per year it'll still be easier to pull in the latest BEAM and Elixir versions and play around than figuring out if Homebrew has the new version, and if so, whether it installs nicely over the old or not.
I don't think I've got the LSP running, might check tomorrow. It's OK, the IDE autocompletes some things and for me development basically happens in the REPL and then gets pasted into tests and the project anyway. And iex has good autocomplete and help functions and so on.
Edit: A hurdle might be to install the GUI libraries needed for Observer, but you'll probably be able to search out an incantation for your operating system once you're into doing stuff with process trees.
I think the Python dev style I adopted can't be easily ported to Elixir. In Python, I rely heavily on an LSP because I want to fiddle with a lot of functions/classes located deeply in libraries. In VSCode, I simply press CMD and click on any function (or in Neovim, I `gf` or `gd` it). I thought it'd make even more sense in Elixir because apparently everything is a module. Am I missing something? How do you use iex efficiently?
Sorry you are having so many issues. That has not been my experience with elixir-ls either locally or in Codespaces. Just want to say that when you do get it working, it does indeed have that and many more features. If you are interested, the Elixir slack is very active and helpful and there is a #language-server channel.
Thank you! It seems the Elixir LSP does provide useful features... I'm worried that even after getting it working on my Mac, the same troubles happen again when I move to remote machines. I'll try the Elixir slack!
Yeah, the LSP situation remains a sore point, which is deeply unfortunate. One of the big reasons I like Gleam! Luckily, there are new contenders popping up to hopefully solve the issues with elixir-ls: try https://github.com/elixir-tools/next-ls or https://github.com/lexical-lsp/lexical. They might give a better experience. In fact, the creator of Elixir recently started directly sponsoring next-ls, so it's probably a safe enough bet.
> By the way, the official Elixir website recommends using Homebrew to install it. But almost everyone in the Github issues and comments says ASDF is the way to go.
The Elixir website is right. Just use Homebrew until you find a real need for asdf or similar tools. It's far simpler.
asdf (or mise[0]) is merely a way to manage different runtime versions between various projects, you would use it the same way as one might use rbenv/rvm, nvm/n, pyenv, or even Docker/nix, and so on. You don't need it until you have several ongoing projects requiring different runtime versions. If you reach that point, great! It'll be worth the effort then, and it isn't difficult.
Personally, I just use Homebrew Elixir for easy ad-hoc access to iex/livebook. If I truly need reproducible environments, devbox[1] (a sort of nix wrapper) is nice and extremely straightforward.
Tl;dr: Just use Homebrew. If your requirements expand beyond that, you probably have far more challenging problems than installing asdf (or whatever).
Thanks! I just checked out devbox, it's so great to have a more user-friendly nix for people like me who don't want to mess with nix!
Also, great point about Gleam! What I like about it, other than types and JS compilation, is exactly the tooling! The devs clearly understood that tooling is extremely important these days and have focused on that. If it's truly inter-operable with Elixir/Erlang libs, I'll probably just use Gleam instead.
People have to realise these are mostly for hobby. It is really hard to get these working with other libraries.
Take explorer, it’s a mess trying to implement dplyr verbs in elixir. Anyone trying to use it is going to hit its limitations sooner or later. I tried migrating to it from polars but it is too frustrating.. gave up after some time.
Why will people use half baked libraries instead of python ? I will stick to Keres/pytorch, polars, etc
> People have to realise these are mostly for hobby. It is really hard to get these working with other libraries.
I have been dealing with this type of feedback for almost 20 years at this point: "This library doesn't work for my needs, so the conclusion is that it is hobby/not useful/etc for everyone". In your experience, we are not there yet (and we may never be), but the generalization does not hold.
Take Explorer, it was the first library of the bunch to be used in production because it was created by Chris Grainger to write data pipelines at his company. And the blog post does include the link to a talk on production usage.
Other projects, like Nx, started as an experiment, but nowadays it is used in production quite a bit and we spend a lot of the time refining it based on feedback we receive.
No one is claiming those libraries are complete. It is still pre-1.0. But we claim to be evolving and improving based on feedback. So if anyone wants to try something different than Python, and you can afford to engage, give feedback to maintainers, and even contribute, then you will hopefully have a good time and be closer to running it in production too.
Thank you Jose, for your 2 decades of relentless work. I have not disparaged any maintainers or their work. I get it from your comments that these projects are in early days of adoption and development.
From my perspective, just because Chris Grainger does data pipelines in production does not prove the project is for everyone. It means it worked out for him.
May be seeing more updates, papers, interviews, podcasts about their usage in production will change things.
For now, my assessment is everything is wonderful till one hits a limitation of these libraries. If it works out, its wonderful, if not its like neither here nor there.
That's precisely my point. Because Chris runs it in production, it doesn't mean the project is production-ready for everyone. And the fact you ran into roadblocks, doesn't mean they are "mostly for hobby" or are "half-baked". Your initial comment lacked this nuance and rushed into generalizations, which I did my best to clarify.
I’ve been considering using elixir for a while. With these libraries, it’s now entirely feasible to move my backend over to elixir, getting rid of 95% of my python code in the process. The tasks leftover that still need python can easily each fit in their own <100 line file and work just as well using other languages to process the data. Most of it needs to be run on a gpu anyways so it’s already sandboxed wrt deployments
It’s not an all or nothing game, and I rue the day that I decided to do everything in python just because I needed a couple LoC to call a python module.
At this point is isnt trying to convert a happy python user like you. Rather to give tools to teams whose app is already in Elixir and the devs knows Elixir. Instead of bringing in Python to the mix you can use Elixir
Elixir dev you are describing and we use Python for ML stuff because it’s easier to work with the more documented frameworks. These libraries just aren’t “there” yet, and I think it’s important to be honest about that. I’ve tried extending/converting to them, and it just wasn’t worth it yet.
> These libraries just aren’t “there” yet, and I think it’s important to be honest about that.
Exactly. These projects are still during early adoption stage.
While it is production-ready and many are doing so, getting there requires improving documentation, discussing roadmaps with maintainers about a feature you may need (perhaps even contributing it), giving presentations (at your company or at events), and so forth. This can be exciting to some, but a distraction for others. There is certainly plenty of work ahead.
To be fair to the creators of “half baked” libraries, you’re comparing our work to frameworks with several years head start with the backing of trillion dollar organizations (excluding polars). To have convinced ANYBODY to successfully make the switch is, to me, both somewhat of a miracle and an indication that there is a legitimate market for this work. I think it also dispels the hobby claim
As for your issues with Explorer, perhaps it is not the library, but your knowledge of Elixir that was a little too half baked? ;)
Telling someone they’re ignorant probably doesn’t get them to use your tech. Not the person you’re replying to; I use elixir everyday at work.
The biggest barrier to adoption of these libraries I can see is that there is not a wealth of resources to solve the actual problem the person is experiencing.
Googling “dyplr verbs in python” there’s a towards data science post that’s the 6th result. It has step by step code examples to get you started with what I guess the person is looking for.
Googling “dyplr verbs in elixir”, you get the explorer library’s GitHub page and docs as results. I see there’s an example on the first page, but docs present as a high barrier to entry. For better or worse, hand-holding helps get people using the thing, and there’s a SEO-fueled ton of python hand-holding resources out there.
I’m sure you’re aware of these things as the author of the book mentioned, but buying a book is a step for someone who’s already sold on learning the language/framework. Having google-able content is much more useful to get people in the door and in the day-to-day work looking something up.
Until something “cool” and well-known is made with Nx/Axon and friends, they’re gonna be seen as second-class to most people who aren’t elixir diehards. Somehow everyone in elixir-land recs these, but never has anything they actually use them for.
There’s also the switching cost. Most Python ML compute is actually done in C/off-loaded to gpu, so its speed is ok and not greatly constrained by the language choice. Python does have multiprocessing stuff which, while annoying in my experience, makes parallelizing stuff possible. Again, thinking from an outsider’s perspective, there doesn’t seem to be a “must have” reason to use elixir when they’ve already been working in python.
I appreciate your response. I should not have replied in the manner that I did. I think there are many ways to make the argument against using these libraries, and all of the maintainers would agree it doesn't make sense in many situations. I am fine with these arguments, but I don't think it warrants referring to the work we've done as half-baked or hobby projects. Even if not the intention, I think it implies a lack of effort and professionalism which comes across as more disparaging than simply pointing out the gaps.
Sean Moriarity, please use your elixir knowledge to write a user defined function or a plugin in explorer. It's easy to attack other people and assume they are ignorant.
It has always been possible to do custom transformations via Series.transform. And here is a PR adding it to dataframes: https://github.com/elixir-explorer/explorer/pull/912 - the PR uses existing APIs, it was possible before, it just makes it more convenient. I am sure more will be added later
The downvotes are for the attitude. If you expected all alternative software options to be strictly better than their replacements we would never get anywhere in terms of DX. Someone has to be brave and be an early adopted and put in the sweat to get shit to work and this guy is just taking potshots instead of making pull requests.
If you suggest what they're correct in and how the replies support your theory, I think the downvotes would be rectified and everyone involved would leave with a better understanding of the topic.
That’s ok, truth is bitter. People may fall for hype sometimes. Maintainers of the project can refute what I am saying but they won’t as this the truth.
- In current explorer one cannot write user defined functions, plugins.
- They don’t even have it in roadmap.
- Also the exposed functionality is always lagging when compared to polars.
- elixir, nx, explorer,distributed, etc nice keywords - reality is it is still early for them. Better to stay away for production apps in case of ai/ml unless people are connecting to OpenAI and writing chat apps for which they won’t need any of the above except elixir.
And there's also a scale to 0 story for when you're not using that GPU at all: https://github.com/phoenixframework/flame
1 language/toolchain. 1 deployable app. Real time and distributed machine learning baked in. 1 dev can go really far.