This is a pre-release but it can be installed using `wsl --update --pre-release`

Pyarrow from 0.14.0 to 14.0.0 allows for arbitrary code execution

IMHO `--fixup` commits and `--autosquash` rebases make rebase much easier.

Interestingly, Dask runs out of memory on many of the tasks of the benchmark.

I had similar issues with the Dask scheduler a few months ago. The docs say it encourages depth first behavior in the computation graph but in my case it kept running out of memory on a large ETL task by first trying to load all the input files into memory before moving on to the next stage.

I agree with you on the state of podman compose.

I just wanted to clarify that you can run docker-compose with podman in _rootless_ mode. This can be done by running podman's systemd user service.

FWIW I actually tend to run podman compose inside a podman container. This is so I can containerize the integration tests which orchestrate the app. It's a useful pattern - one I think should be a lot more widely used for several reasons. The systemd service wouldn't work in this context.

I could maybe use some entrypoint magic to run a server when the container starts just so I can use docker compose but still...eww.

Running a podman systemd service might suffice if it was something that could be installed with a snap of two fingers on every environment but if it means fscking around with service files it's definitely not something I'd want to add to a "set up a development environment" README.

I see, interesting. I guess you are saying that systemd service wouldn't work because it's not available inside the container.

FWIW it's possible to run a rootless podman container with a working systemd inside the container. I've near tried running podman in podman using systemd though.

You can use docker-compose with podman using podman's docker compatibility API.

See my other comment in a recent thread [0]

[0] : https://news.ycombinator.com/item?id=37661056

A trick I have used sucessfully is to use docker-compose with podman as the engine.

The way I do it is by starting podman's systemd service which exposes a docker compatible API over a Unix socket.

Then export DOCKER_HOST pointing to the socket and you can run docker-compose with podman transparently.

I think it's this bug https://github.com/microsoft/WSL/issues/8696

A real nightmare.

There are several issues in the repo related this bug, here's another one with over 500 comments:

https://github.com/microsoft/WSL/issues/6982

Almost 1000 total comments between both of these and basically no acknowledgement from microsoft.

This release improves memory management (automatic reclaim) and network compatibility.

Fantastic! Lack of memory reclaim is one of my main gripes with WSL, so I'm really pleased to see this.

You can dump caches in current wsl2, you just have to dump a value into a /proc file to trigger it

And disk reclaim with sparse vhd

So this is WSL2 2.0?

I'm confused on this as well. (I almost said I'm confused on this 2)

Joke aside... I am still confused.

I feel like they're competing with USB version numbers.

You can host your own swarm of servers apparently [0]. I would be curious to have a ballpark estimate of the finetunning performance of a "private" petals cluster.

[0] https://github.com/bigscience-workshop/petals/wiki/Launch-yo...

I think if you run a cluster in a trusted environment it should be more efficient to use ray or something similar