Using Go in anger for a few years has made me realise just how much a good package management solution adds to an ecosystem, and just how difficult a bad one makes life.

I've been flip flopping between Rust and Go and landed on primarily Rust due to not liking how imports are done by URL in Go. Probably works for Google, but it doesn't make sense for most use cases where you want to handle major version changes.

That’s fine when you are working within one language, but many of these people work on packaging across language boundaries (executables and libraries)

If you don't embed a c lib, or don't need to setup a database.

I can imagine FFI being tricky. But what’s tricky with regards to databases in this context?

If your app depends on postgres, the current habits to state in the readme you have to install it and setup it yourself.

That's not packaging.

Goog packaging deals with that for you.

I’m not sure I follow… If you write an application in Java/Javascript/Python/C/C#/C++/Racket/Common Lisp, isn’t it the same story there? I.e. that you’re expected to provide the DB yourself.

If one desires a convenient solution there’s always Firebase, DynamoDB or even hosted PostgreSQL solutions.

For local development one can spin up a Docker image for the DBMS.

Yes, that's why packaging is more complicated as soon as you use more than your own language.

Go solves only one problem, albeit very well.

I imagine the conference swag consists of branded tote bags, boxes, baggies, bins, pouches, bottles, backpacks, and laundry bags.

All made using YARN!

Also eggs and wheels, cause Python.

Maybe we will finally get some protocol standardization in package distribution.

If you think about it it's kind of crazy that we are reinventing the wheel with every language, OS, game store, extension marketplace etc.

A lot of Linux enthusiasts get lost in the idea that there should be one and only one package manager. But actually it's more that there should be a standard protocol for package discovery, distribution and updating; and package managers can just be an interface to those.

I think you’re overestimating how generic package management is. They’re very tightly coupled to their respective languages and build systems.

I worked on the problem for two years. I'm not overestimating anything; it's just that the wrong targets are being set.

Like I said in my post, it's a "single package manager to rule them all" is the wrong goal. What you want is a common protocol, and multiple implementations of it.

"I want an app store" = spin up your favourite PackManProtocol server, configure it to your needs, done.

The basics of package management are all very similar and can be covered by core use cases. The solution would need to be extensible enough to be usable beyond those. But having restrictions can also be good for an ecosystem (preventing weird-feature creep).

> But actually it's more that there should be a standard protocol for package discovery, distribution and updating; and package managers can just be an interface to those.

PackageKit exists as a standard protocol on top of myriad package managers. Most major distro package managers (apt, yum/dnf, apk, pacman) have a package kit wrapper, and UI tools such as Gnome Software or KDE Discover exist that use this abstraction.

The missing piece here is that it still requires packages to be built in all the different formats that these abstracted package managers require.

Exactly. And that one correct true best winning choice can surely only be rpm.

I always wonder how much the split between .deb and .rpm had to do with the names. They serve the exact same function to the point where there is translation software from one to the other and vice versa, but the names themselves promote the distros they were designed for. Is that the biggest reason for the split? Of course the debian project managed the development of .debs and Red Hat managed the development of rpms, so control of that development path was probably another reason. But this is one split in the linux community that really feels unnecessary.

.deb predates RPM's by a few years, so it's probably more then just the name.

While they accomplish the same thing, RPM's are actually pretty different from .deb since they're binary files. There's more metadata encoded in the package themselves, as well as a built in signature. .debs are, at their core, still just fancy tarballs organized in a specific manner.

Honestly though, RPM's are kind of a mess and pretty difficult to implement without using one of the reference programs, along with containing a lot of extraneous info that you need to fill out. There's a great blog post out there describing someone's attempt to create RPM's from scratch and their gradual descent into insanity. Though, the extra metadata has it's features (better search, dependency resolution, etc).

It has probably been 20 years since I did anything with deb and rpm, but I think this still holds true:

The deb and rpm formats had a very different philosophy to handling dependencies: the deb format relies completely on package names, while the rpm format also allows you depend on the existing of files (or other things?). That implies that the doing/deb ecosystem only works if you manage the entire OS, while rpm (at least theoretically) allows you to use rpms in a system managed in some other way. Of course that capability in the package format itself did not mean people used that.

The file formats are also slightly different: dpkg/deb is completely build using tar and ar, while rpm is a custom binary based on cpio. That also made the deb format flexible enough to not have needed any changes in 30 years, which is pretty amazing.

The source formats have a lot more differences, but I don’t know enough about those. From what I remember srpm used to be more powerful, but Debian’s source format has caught up by now.

A nice side effect of only Debian using dpkg/deb, at least historically, was that when you got a deb file it was guaranteed to work on your Debian system, but for an rpm you could never predict if it would work on your RedHat/SuSe/Mandriva/TurboLinux/etc. system.

http://xkcd.com/927/ sorry, but I had to

Upvoted because it needs repeating. And also https://xkcd.com/1053/

I'm not sure that's what people mean by package management.

For example if a configure script has a dependency on a library... then what? It doesn't help me find and install the right version of that dependency, does it? In my experience it just says 'dependency not installed'.

I think it's meant as a joke, but for many of us, there's some truth to it.

If you're missing a dependency, you'll need to download, configure, make, install it.

> If you're missing a dependency, you'll need to download, configure, make, install it.

Right but people want their package managers do this for them. That’s the whole point.

That's what indicates to me that it was intended as a joke, yes.

That often doesn't work on windows. Autotools are brittle at the best of times even on the same platform. Makefiles aren't actually portable (there are many flavours of make). none of that is standardised in any way, no software that I have installed in the last 10 years has had it as its install instructions for anything other than development.

It also doesn't handle dependencies, if the library uses libfoo@v0.0.1 and I have libfoo@v0.0.2 installed, this will happily attempt to continue. This isn't a solved problem, by any stretch of the imagination.

That's just for building though, isn't it? Package managers these days deal w/ network, registries, version management, transitive dependency resolution, etc. Not to mention Windows.

And it's showing its age!

Your suggestion doesn't install dependencies, doesn't check for out of date software, and doesn't work on windows. Those all seem like fairly major issues to me.

it's not a useful standard if it does not work out-of-the-box on a native windows install

Great, I have a message about py, -m, --user, shiv and nuitka that I've been repeating for years but really need to reach a broader audience. That's going to be a great talk to give, and 20 minutes is the perfect format for it.

Python packaging would be way less painful if people had been given those key informations. Not painless mind you, but from missing a leg to balls itching.

It would be grest if they offered group discounts for Python packaging solutions developers.

it seems to be pay what you can afford.

Interesting that also the Julia Package manager has a presentation.

Client implementations typically need to be deeply integrated with the host PL/OS but I think we can create a shared package registry with a protocol that sits on top of IPFS.

A single tool for publishing to the registry would push the package to IPFS, get the returned CID (hash of the package) and store the hash with some package meta data in a blockchain. Storing the hash in a blockchain would provide confidence the package has not been tampered with assuming no 51% attack and that client implementations always verify the hash against the package bytes. By making the hash immutable and linked to the content address of the package we have more confidence in the integrity of our packages.

Then we add an HTTPS bridge so that client implementations can easily migrate to consuming packages via this new distributed registry.

I am exploring this now so if anyone is interested in this kind of distributed language-agnostic package registry drop me a line at muji [at] tmpfs.org.

Nix is already most of the way there with content-addressed packages. An IPFS project is also in the works using this feature. A blockchain is unnecessary because reproducible packages can be checked with `nix build --rebuild` and stores are signed by default. A bridge is unnecessary because Nix wraps language-specific package managers when building and does not allow network access in its sandbox.

Thanks, i will take a closer look at nix!