2014: Python is a pain in the butt to manage packages and dependencies, how the hell am I gonna deploy this?
It's still a mess 10 years later unless you live deep in the ecosystem and know what third-party solutions du jour to manage that complexity. At least we have Docker now.
Also let's not forget the Python 3 migration fiasco that lasted ~2008-2018 and I still find myself porting libraries to this day.
I haven't used Python in a personal project in 10 years because of these painful paper cuts .
At least for Python dependency management, I'd hardly call it a mess these days and yes it was a horrible 10 years ago. The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days. To the extent that moving between MacOS, Linux and Windows is feasible for most (of my) Python work.
And the Python 2/3 pain stopped being an issue a few years back. I haven't been forced because of library requirement to use Python 2 in years.
Your criticisms WERE valid 5 or 10 years ago. In my experience, the ecosystem has improved greatly since then. I wouldn't use Python for large and shared developer code base but for one-man experiments and explorations, Python is pretty compelling these days in my opinion.
> The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days.
Except Python themselves now recommend the third party, pipenv. [0] Which means that there are multiple authoritative voices saying different things, and that produces confusion.
Now being from 2017 ;)
I suspect no-one has touched that file properly since then because of old scars :(
Part of the issue is there's not a lot of differentiation between most of the "project management" tools (apart from their age, which in some cases explains certain design choices), and the coupling of most of these to their PEP 517 backend doesn't help. There are backends where there is differentiation, such scikit-build, meson-python/mesonpy and enscons for wiring into pre-existing general purpose build systems (cmake, meson and scons respectively), maturin for rust, and flit-core as a bootstrapper, but given setuptools can be used without a setup.py now, I'm not clear on what some of the others provide.
That's why I deliberately described the pip/venv solution as a "de-facto" standard.
Is Python perfect in this regard? Absolutely not but the situation is FAR less painful than it was 10 years ago. The original comment claimed nothing has changed in 10 years. I used Python 10 years ago and disliked it intensely but I've recently (in the last 3 years) used it again and it is simply not true that nothing has improved in 10 years.
I still use python for stuff that never leaves my computer, but in most cases if I know I need to run it on Someone Else's Machine (or even a server of mine) I'll reach for Go instead.
I have a Taskfile for local compilation and running (macOS) and when I need to release I can just say `task publish` and it'll cross-compile a Linux binary and copy it to my server.
Eh, poetry, pipenv, conda, setup.py, GCC and all dev tools to possibly compile some C shite you didn't know you needed? It's a complete shit show still...
I started using Dagster a few days ago, which runs on Python.
As far as I can tell, I have to install the packages with pip. Wait - don't forget venv first. Oh but I'm not supposed to commit the venv, so how is anybody to know what I did? Okay I have to setup a requirements.txt... but now what did I install in the venv? Okay there's a way to dump the venv.. but it's giving me all the dependencies, not the root package I installed?!
Okay there's something called pipenv that seems smart... Oh wait no this Dagster has a pyproject.toml, maybe I should use poetry? Wait poetry doesn't work, it seems like I'm supposed to use setup.py... so what's the point of pyproject.toml.
Okay I put the package into the setup.py. Great now I have to figure out the correct version dependencies by hand?! I don't get it, python is a nightmare.
Compare to .NET:
dotnet add package
On build:
dotnet run # automatically restores packages to a project level scope
I've always wanted to like Python, but I got out of undergrad in 2006 and pretty much the entire first part of my career was the ridiculous Python 3 migration nonsense. Every time I wanted or needed to use Python there was some dependency management nightmare to deal with. For like a decade or so.
I think I'm like you, I have some sort of Python ptsd from those years and don't really care to venture into it again. It sounds like it's gotten a lot better recently, but also after years of using Go and TypeScript I can't imagine working on a codebase without static types.
I'm not so sure this is a desirable state to be in long-term.
While Docker (and other container solutions) does provide solutions to some of the challenges around environment consistency, build artefact management, development environments with linked services, etc., it's also one of numerous de-facto tools that in some ways ends up putting more effort on the end-user to adopt. Less effort than installing all dependencies yourself? Absolutely. More effort (and for no real reward) than a single binary built natively for the target platform? Yep.
This feels like a sort of 'shift right' approach to releasing an application in terms of developer effort. As a developer you can just bundle everything into a container image and now the end user needs Docker and the install is going to leave a mess of image layers on their system. You don't need all the dependencies installed system-wide, that's true, but why can't they just be bundled with the application?
It's also not a great cross-platform tool, as Docker on anything except Linux just hides the requisite virtualisation. It's mostly transparent, true, but is still plagued by platform-specific issues such as write performance on mounted volumes on macOS being ~10x as slow (there's a fairly old thread on the Docker forums that's still open with this as a reported issue, and it does't look like it's going away).
Additionally, while disk storage _is_ (comparably) cheap nowadays, the buildup of image layers that contain endless copies of binary dependencies doesn't feel great. I don't want or need hundreds of copies of libc et al. all with slightly different versions sitting around. Even the 'slim' Debian images are min. 120MB, let alone the 1GB full ones. As Python is on topic for this thread the Alpine images are smaller, but then Alpine image builds for Python don't work with the standard PyPI wheels (I'm not actually sure if this is still true, need to check).
As languages like Python are just source releases, some tooling is definitely needed to make that more palatable in terms of handling releases of Python tools (just big collections of scripts), but being able to build and copy a single binary for a Go application feels great having spent a lot of time practically requiring Docker for release Python tools.
And all of this is really just getting us back to where we were years ago, where it was actually feasible to release a native binary for a target platform. The difference is that Go's tooling around related areas such as dependency management and cross-platform compilation is more standardised, and as a result the developer experience for this is much nicer than with other comparable languages that also compile to native code.
Arguably a lot of the development decisions around how to manage and release tools comes down to how good the experience is for developers. Docker ends up being favourable for Python because there's no 'standard' Python way to bundle applications with their dependencies and Docker basically solves this problem, even if in a messy way.
Does Docker have its place? Definitely. Do I hope it sticks around in favour of improved cross-platform developer tooling? Absolutely not.
As opposed to writing everything from scratch? Python has a huge standard library. Making it bigger to avoid dependencies would add more bloat to the distribution.
I have a nice way of testing a language - I download a couple of projects written by beginner / mid-level developers and not having commercial dependencies and see how much effort it takes to get them running.
Python is one of the worst performers (and Java is shockingly bad too, although a lot of that is down to the way the JVM/language have been mismanaged). At least unless you're comparing it to very low-level languages (VDL, C).
> and Java is shockingly bad too, although a lot of that is down to the way the JVM/language have been mismanaged
This is interesting and not what I would expect. Generally to get a java app running from github you'd install the correct JDK and that should be about it. Many projects will use maven, which will know how to obtain dependencies. Could you describe a typical issue?
Dependencies requiring you edit some random XML file somewhere on your machine to get them working.
That and the whole JVM fragmentation and not being able to touch Oracle because it's Oracle and they'll use it as an excuse to sue you (at least that's the impression I had, the entire lawsuits thing is why I left Java back in the day).
You can say the same about other package managers, but it's not very often that I have to add another source to apt unless I'm running in a locked-down environment.
> Dependencies requiring you edit some random XML file somewhere on your machine to get them working.
No it doesn't. That's only if you're dealing with private maven repos and authentication etc. You don't need to do that at all for the use case in this thread, running a project from github.
Like the C and C++ compiler fragmentation, each with their own flavour of ISO support, UB and language extensions?
Or the Go compiler fragmentation, where only the reference compiler supports everything, tinygo a subset, and gccgo left to die in Go pre-generics era.
No but you can just use ./mvnw instead of installing it. I have never had to install gradle as a package on my computer in my entire lifetime. I always used ./gradlew.
My usual test for choosing a new programming language, is to see how easy or difficult it is to read documentation without having to open a web browser.
For languages/tools/frameworks/whatever that I'm already using, the main way I compare between them is looking at the ratio of "problems I was able to solve from their official documentation" vs "problems where I needed to use random blog posts (or worse, Stack Overflow) to find the solution".
Makes it a lot easier for me to choose tech for my hobby projects, where I sometimes don't even touch projects for several years because stuff just runs smoothly.
Oracle requiring commercial licensing for their JVM has made it so radioactive that my workplace firewalls "oracle.com" to prevent anyone from accidentally using it.
The docs are surprisingly unspecific for such a mature project, enough that you often need to read source code to confirm your suspicions one way or another. The style has been redone a few different times now, and most of the old stuff is there for backwards compatibility so it's unclear what The Correct Way is to use it (once you step outside the obvious classes and methods).
The source code is like an Escher drawing, I'm sure it makes sense to the maintainer but its logic is incredibly abstract and scattered. From my notes on it ~1 year ago: "I took a dive into Celery source to better understand their native way for inspecting workers... holy shit. Celery is an overgrown jungle of abstractions."
It's honestly kind of surprising to me that Celery is still the undisputed #1 message queue abstraction library. I guess it's a simple enough problem that companies using Python at scale roll their own. I wonder what Instagram uses instead of Celery.
I found this [1] to be a useful comparison between queues in Python, if anybody has found a similarly-terse and exhaustive one that's more up to date for 2024 I'd love to see it.
dramatiq seems like a true gem. I didn't know it existed. I don't see why this can't just be an immediate Celery replacement for your typical Django project that already uses Celery - with a bit of time spent on refactoring into dramatiq.
Its powerful but good luck reading the source. Its a bit of a tangled over-engineered mess at this point and the reason there are a number of "newer" libraries to try and replace it. I could not recall specifics since it has been maybe a decade since I last used it but it works until it does not and then its incredibly hard to debug.
Its been a decade and maybe its gotten better but that sentiment is why a number of people do not want to use celery. I would also add that it was created in a different mindset of delayed job and I believe there are better patterns these days without as much complexity.
I only dabbled with Python occasionally in college, and the lack of types was my biggest obstacle. Even reading the official documentation of libraries, I still couldn't figure out what I should expect from whatever function I was calling.
I recall the Selenium library I was using as a particularly gnarly offender -- pretty much every function I called, I had to debug or `print(typeof(x))` to figure out what I'd gotten back.
yeah ... i've become far more disenchanted with loosely typed languages. It's amazing to be able to quickly script things together and focus on the happy path in one-off scripts and bash-replacements but ... in a huge system when reasoning about what's going on -- I find it far more rewarding to know exactly what the type of a thing is and what it can do, etc., etc., and here with Python I rely on the editor a ton and I don't really like that.
2014: Python is a great language, but there are a few pitfalls
2016: Python is a good language with the right IDE, tooling, and process. The people are pretty cool though.
2018: I like python, but I wish more people used type annotations.
2020: You know, metaclasses are freaking awesome! They saved me so much work!
2022: Why can't people code the most obvious solution in python?
2024: Celery! Jesus H. Christ! What were you thinking?!