I am very interested in improving the user-experience around precompilation and performance, may I ask why you are creating a sysimage from scratch?
> I would opt into prebuilt x86_64 generic binaries if Julia had them
The environment varial JULIA_CPU_TARGET [1] is what you are looking for, it controls what micro-architecture Julia emits for and supports multi-versioning.
As an example Julia is built with [2]:
generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)
I have a monorepo full of Julia analysis scripts written by different people. I want to run them in a Docker container on ephemeral Linux EC2 instances and on user Windows workstations. I don't want to sit through precompilation of all dependencies whenever a new machine runs a particular version of the Julia project for the first time because it takes a truly remarkable amount of time. For the ephemeral Linux instances running Julia in Docker, that happens on every run. Precompiling at Docker build time doesn't help you; it precompiles everything again when you run that container on a different host computer. R and Python don't work like this; if you install everything during the Docker image build, they will not suddenly trigger a lengthy recompilation when run on a different host machine.
I am intimately familiar with JULIA_CPU_TARGET; it's part of configuring PackageCompiler and I had to spend a fair amount of time figuring it out. Mine is [0]. It's not related to what I was discussing there. I am looking for Julia to operate a package manager service like R's CRAN/Posit PPM or Python's PyPI/Conda that distributes compiled binaries for supported platforms. JuliaHub only distributes source code.
My point is if you set JULIA_CPU_TARGET during the docker build process, you will get relocatable binaries that are multi-versioned and will work on other micro-architecture? It's not just for PackageCompiler, but also for Julia's native code cache.
It worked! I was able to drop the Windows install on a standard GitHub Actions worker from 1 hour to 27 minutes. Here's what worked:
ARG JULIA_CPU_TARGET="generic;skylake-avx512,clone_all;cascadelake,clone_all;icelake-server,clone_all;sapphirerapids,clone_all;znver4,clone_all;znver2,clone_all"
ARG JULIA_PROJECT=[...]
ENV JULIA_PROJECT=[...]
RUN julia -e "using Pkg; Pkg.activate(\"[...]\"); Pkg.instantiate(); Pkg.precompile();"
What I got wrong the first time: I failed to actually export JULIA_CPU_TARGET so it would take effect in the "Pkg.precompile()" command. In reality, I hadn't correctly tested with that environment variable set at all. I was only correctly setting it when running PackageCompiler.
Thank you so much for this! It's too late for me to edit my original post, but cutting the install time in half is a major win for me. Now it only needs to precompile, not also compile a sysimage.
... Actually, this only worked for Linux. My Windows container is back to precompiling every time, again, and Windows was the slow one that I wanted to fix in the first place. I had to revert this. Back to the drawing board. I wish Julia would print some diagnostics about why it decided to precompile again.
That was the very first thing I tried, and I couldn't get it to work, but I'm sure I am doing something wrong. Everything seemed great at build time, and then it just precompiles again at runtime, without anything saying why it decided to do that. I'll give it another shot if you say it should be working. The PackageCompiler step is the longest part; if that can be removed, it'll make a big difference. I'd rather be wrong and have this working than the other way around :) I'll report back with what I find.
We use RR a lot with Julia. It only gives you a GDB view of the system, but it can work with any interpreted or compiled language.
Things that don't work are drivers that update mapped addresses directly. An example of this is CUDA in order to replay one would need to model the driver interactions (and that's even before you get to UVM)
Another great thing is that RR records the process tree and so you can easily look at different processes spawned by your executable.
Shameless plug: https://undo.io does everything rr can but can also work with drivers that update the process memory (or unrecorded processes or even hardware). If you need that kind of advanced usage, check it out.
(Unlike rr it's not open source though - sorry! We have lots of programmes working on it full time and they insist on getting paid every month :)
Changing the `function foo()` in the "Usage: descend" section to `function madness()` sounds like a pretty good/funny idea.
@descend madness()
would be a pretty good reference to the original text!
Positive: it would also let people know at a glance that things like `rand() > 0.5 ? Int64 : Float64` are indeed madness.
Potential negative: some people may not consider it super DEI-friendly? As a neurodivergent person, I'd just find it to be just a lovely reference though.
> And dealing with bugs in LLVM is basically a no-go, I've seen this happen in the Julia ecosystem as well.
As one of the folks dealing with LLVM bugs in the Julia ecosystem.
Yes it requires a distinct skillet different from working on the higher-level Julia compiler and yes it can sometimes take ages to merge bugfixes upstream, but we actually have a rather good and productive relationship with upstream and the project would get a lot less done if we decided to get rid of LLVM.
In particular GPU support and HPC support (hello PPC) depends on it.
But this is also why we maintain the stance that people need to build Julia against our patchset/fork and will not invest time in bugs filled against Julia builds that didn't use those patches. This happens in particular with distro builds.
Yeah I often describe Julia as a Lisp in sheep's clothing.
Or as the m-Lisp promised to us :)
I chuckled when I read:
> The way that common Lisp systems produce executable binaries to be used as application deliverables is by literally dumping the contents of memory into a file with a little header to start things back up again.
Which is pretty much of Julia's sys-/pkgimages work. Pkgimages are an incremental variation on this idea.
One of the novelties in Julia is the world-age system and the limits on dynamisim it introduces on eval.
Yes that is precisely what was fixed, essentially the thread local storage that Julia was expecting were not setup and thus calling the runtime from a foreign thread would cause a crash.
This now enables to dynamically add and remove threads.
How closely tied is this to Python?
The need for reactivity is what drove the development for Pluto.jl, but it would be nice to have something like this for IJulia.jl as well.
For this project, Python is a hard requirement, though it's possible the approach may be applied (after significant effort -- Python took me ~3 years and counting) to other languages / runtimes as well.
How closely tied is this to Python?
The need for reactivity is what drove the development for Pluto.jl, but it would be nice to have something like this for IJulia.jl as well.
> I would opt into prebuilt x86_64 generic binaries if Julia had them
The environment varial JULIA_CPU_TARGET [1] is what you are looking for, it controls what micro-architecture Julia emits for and supports multi-versioning.
As an example Julia is built with [2]: generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)
[1] https://docs.julialang.org/en/v1/manual/environment-variable...
[2] https://github.com/JuliaCI/julia-buildkite/blob/9c9f7d324c94...
reply