I also have complaints about the package manager. A language as new as Flutter shouldn't have significant dependency conflict hell as it is a solved problem. Npm/yarn, cargo, Go are all able to (in most cases) handle conflicting upstream dependencies. In other words, if package A required version 2 of C, and B requires version 1, the code can still compile without any user intervention (such as shading) barring some corner cases. This is the biggest difference between the new generation of package managers and the old pip/gem/maven type. For some reason, Dart's authors decided to use the traditional Java style package management algorithm (which in the worst case, on paper at least, requires NP time constraint solving but this is rarely an issue in practice). I have wasted days trying to get an obscure third party dependency n levels up the chain to build so the codebase can successfully compile.
I strongly suggest aspiring dependency manager authors read these
before building yet another broken package management solution. This is a huge problem in the Lisp community too but their problems are worse. Racket doesn't have lockfiles. Same problem with quicklisp. The alternative replacements like Racksnaps don't support conflicting versions either. As far as I know, CLPM is the only modern dependency manager in the Lisp world though it is very poorly documented, barely maintained, and installing it is difficult. If you go to the racket mailing list, nobody seems to be interested in tackling the problem or even acknowledging that it exists.
Hm... I am kind of collecting all information I can about package managers for a few months now... pub is one of the best I've seen.
Cargo/Go/npm, as you say, will try to "shade" conflicting versions, which is basically a gamble. It may or may not work at runtime because data exchanged between different nodes in the tree of dependencies may have conflicts that make them incompatible... so having module A depend on version 1 of X, while module B depends on version 2 of X, and then A and B exchange objects that depend on which version of X they use will cause issues... this will blow up at compile time in Rust, at least, if you expose X in the API of A and B... but even if you don't, it's still possible for a conflict to occur if the exchanged information is in the form of serialized data or generic collections.
Pub selects one version of each library. It has great tools like listing the currently latest versions of every dependency, or the currently updatable ones given your version constraints, it addresses Dart SDK bounds of each package and it just works greeat. Can you give an example of when it didn't work for you?
EDIT : in the Lisp world, the biggest package manager is Quicklisp[1] as far as I know... and it works decently! I think Ultralisp[2] might be somehow better because it updates indexes faster or something.
I unfortunately didn't keep great records at the time (probably because I was so pissed off), but my app [1] has gone through some dependency hell on more than one occasion. I've more or less stepped back from the project but the thing that sticks out to me the most is when I left the project for several months, then tried to re-clone and re-build the app to fix a bug. I ended up having to change at least a few depenencies and even send in an upstream PR to get things working (thankfully accepted quickly) before I could compile because something had somehow changed in those few months. Fixing it took a couple days of effort, which for a toy side-project almost killed it on the spot.
In my experience, Flutter is the worst cross-platform mobile development tool; except for all the others.
Build a project with enough dependencies and there will always be dependency issues which you will need to fix by forking upstream. For Rust and Go the package manager can trace the codepath and identify conflicting type exports.
> In other words, if package A required version 2 of C, and B requires version 1, the code can still compile without any user intervention (such as shading) barring some corner cases.
I never understood why you would want this to ever compile, especially as the default. There is no way to guarantee that two versions of a library loaded into the same program will work together, so of course you want the compilation/package resolution to fail if this happens.
Now, in some rare cases this may not be a problem, so it would be nice to have an escape hatch to permit this highly unusual use case, but it certainly can't be the default in any sane technology stack.
> Go are all able to (in most cases) handle conflicting upstream dependencies. In other words, if package A required version 2 of C, and B requires version 1, the code can still compile without any user intervention (such as shading) barring some corner cases.
Different languages place different constraints on package management. For Go:
1. The language is structurally typed. If A gets a value from C 2.0 and passes it to B which expects a C 1.0 value, as long as the interfaces are structurally compatible, it will compile. (Whether it runs correctly is a completely open question. In principle, it won't because C is explicitly saying that 1.0 and 2.0 are incompatible. But there's nothing in the language to prevent a value from one version of C being passed to a function from the other version of C.)
Dart (like most languages) is nominally typed. If you have a class named Foo defined in two different libraries (or different versions of the "same" library), they are considered different, unrelated classes. If Dart were to allow multiple versions of the same package, then users will run into compile errors like "Expected a Foo (1.0) here but got a Foo (2.0)."
2. Code size is relatively less important for a primarily server-side language. Most Go developers don't really care how big their executable is as long as its within reason. Want to compile in five versions of some package? No big deal.
Dart is designed for client-side mobile apps, both native and compiled to JS. That means small code size is an absolutely critical performance requirement. Silently allowing multiple versions of the same package even when it's possible to find a single version that satisfies all constraints would bloat applications for no benefit.
This was historically a problem when people started trying to use the Node/NPM ecosystem in the browser. NPM freely gives you multiple versions of packages and the end result was large apps that weren't friendly to running in the browser. NPM added shared dependencies later to try to mitigate this, but now you've got two ways to manage dependencies and the extra complexity that entails.
3. Go chose to bake the major version number directly into each import in the source files. This makes imports unambiguous in the presence of multiple major versions of a package. But it means that upgrading a package to a new major version is a sweeping transitive source code change even when the new major version is in fact backwards compatible, which is the common case.
For Dart, we wanted users to be able to rev versions more easily than that and keep version management outside of source code.
> Dart's authors decided to use the traditional Java style package management algorithm (which in the worst case, on paper at least, requires NP time constraint solving but this is rarely an issue in practice).
Pub is most closely based on Bundler (which in turn informed the design of Cargo), and not Maven/Ivy/Ant. You're correct that version constraint solving is NP-complete. Fortunately, we have a state of the art solver [1] and it generally finds solutions very quickly. When it fails, it tends to fail fast and report helpful errors.
It's not perfect but there is no silver bullet for code reuse, and package management is fundamentally about code reuse. Code reuse is hard and anyone claiming they have a solution that makes it easy is most likely sweeping some unsolved part of the problem under the rug.
I strongly suggest aspiring dependency manager authors read these
https://research.swtch.com/version-sat https://pnpm.io/faq
before building yet another broken package management solution. This is a huge problem in the Lisp community too but their problems are worse. Racket doesn't have lockfiles. Same problem with quicklisp. The alternative replacements like Racksnaps don't support conflicting versions either. As far as I know, CLPM is the only modern dependency manager in the Lisp world though it is very poorly documented, barely maintained, and installing it is difficult. If you go to the racket mailing list, nobody seems to be interested in tackling the problem or even acknowledging that it exists.