I'm strictly a "scientific" programmer. I've used multiple languages in 40+ years, but only two lasted more than a decade: Turbo Pascal and Python. As Python has passed the decade mark for me, I sometimes wonder why I'm still using it and what would make me get sick of it.
Some of my reasons for choosing Python in the first place have been mooted. Python is free, and multi-platform, but so is everything else nowadays. I think that battle is over.
For my use, packaging has not been a big obstacle, maybe because I tend to use a relatively small handful of established packages. I don't remember needing a package that I couldn't install with pip.
Easy to learn? Yes and no. I think a beginner can ease their way into Python and get productive, if they start with basic scripting and progressively adding a few language features. On the other hand, some features are simply over the heads of beginners, making the code found in packages practically unreadable. But I've never found any language to be better in this regard.
Fast? Sure, with numpy in its domain of use. Or writing your own C libraries. However, I think a beginner using numpy will run into roadblocks that require help from an experienced programmer, who can visualize what the array operations are doing under the hood. So, writing fast code isn't easier to learn in Python than in any other language.
The editor should have a final option, that turns your code brown if it's just so bad that nobody should ever have to read it, including yourself.
I now use python, but Python is not even half the language Mathematica is for scientific computing.
If only Mathematica was free, I think it would become the defacto language for scientists.
The most important feature of Mathematica is that it is functional and not object-oriented. Mathematics is functional, not object-oriented. So thinking in Mathematica is as easy as thinking in maths. Thinking in python is to force yourself to think in an unnatural way.
I think you're regarding Mathematica from a purely "theory/analytics" point of view, however I think you're missing the fact for a lot of scientific computing you want to interact with other things apart from numerics/analytics etc.. I believe that is really where Python shines and why it's also eating Matlab's lunch (more so than Mathematica's). It's incredibly easy to write a gui to interface with e.g. a scientific instrument do some reasonably fast analytics with numpy and possibly even produce publication ready plots. This is not really possible with Mathematica and while you can do this in Matlab, you need several toolboxes which are expensive, but more over are of extremely varying quality (let's not even talk about the horror of writing GUIs in Matlab). With Python, you have all of that for free, with a single tool (remember most scientists don't want to learn lots of programming languages).
Octave/matlab is a little more enjoyable to use for matrix stuff in the REPL though. The game of “is this in Numpy or Scipy and why won’t it autocomplete” is kind of annoying, IMO.
On the other hand, the non-mathematical functionality of Python of course is much better developed!
I like Mathematica a lot, but a couple of issues limit its usefulness for me. First, I'm an experimental scientist, so a lot of programming is in the lab, automating experiments, running tests, etc. Makers of specialized software tend to choose their battles in terms of the application of their product, justifiably, but this means no single app is general purpose enough to serve me in the office, lab, and at home.
Second, unless an organization is enlightened enough to approve a site license, specialized software tends to create a "silo" of people who can use it, and forces it to be used in a centralized fashion. With Python, I can literally install my tools on every computer that I touch -- in the office, multiple labs, at home, etc. It makes it more likely that a tool will be woven into "how I think."
I can also share things and/or encourage others to try them out with minimal friction. Almost as easy as sharing an Excel spreadsheet within an organization that has purchased a site license. Free means free site license.
The friction of approving a license also discourages people from learning a tool by just giving it a try on some trivial project, or taking it home.
Mathematica (and matlab for that matter) is effectively free at most national labs, and it’s still a tiny market share. That’s because scientific computing is rarely just scientists, but often includes engineers, SWEs, system admins, theorists, grad students, and more, solving lots of problems. If you can’t get wide adoption at a national lab, where are you going to get it?
Similar problem with Julia and Fortran even. They all have their niches and are great at certain problems, but Python (and C/C++) rule the overall landscape. in HEP, for example, it’s even the case that a full batteries-included C++ environment (ROOT) has slowly given way to python.
> The most important feature of Mathematica is that it is functional and not object-oriented.
Python has object-oriented features but there is nothing that requires you to use them; you certainly don't have to express every Python program in terms of classes and methods, the way you do in Java. Doing functional programming in Python is common. Is there a particular aspect of doing functional programming in Python that you find to be a roadblock?
It's more of the fact that all popular scientific libraries in python are written in an object-oriented way. That kind of forces you to write your own code in an OO way. I am guilty of this as I am writing a library myself. But users have to use, say numpy, along with my library. It would be suboptimal to force the user to constantly switch between functional and OO, so now my library is OO.
On the other hand, all the code I wrote in Mathematica during my PhD was functional, because it is just easier to write functional in Mathematica.
> It's more of the fact that all popular scientific libraries in python are written in an object-oriented way. That kind of forces you to write your own code in an OO way.
I'm not sure I see why. The object orientation in libraries like numpy or scipy is to provide the user with fast implementations of the kinds of mathematical objects they will need to do computations. But unless you are defining additional such objects in your own code, there's no reason why your own code needs to be object oriented. You can just write ordinary functional programs that use the mathematical objects the library provides.
For example, say you are doing matrix computations. You are of course going to use the matrix objects provided by the library; but your own code shouldn't need to define any new matrix objects. Your own code can just be straightforward computations using the existing matrix objects in the library.
It doesn't matter how libraries are written, in py you can use in functional way. I don't get what do you mean by forcing you to code in OO? I Had used pandas, scikit-learn, numpy, pytorch in production and I had never wrote them in OO way. Are we from same universe?
I believe this is only the language. Not the set of libraries that come with Mathematica. Without those, one cannot do a lot in terms of scientific computation.
I often like to compare Python and Mathematica to English and Latin.
English is not really a beautiful language - it’s a wild mess of different influences, but it’s easy to get to a basic level of fluency as there are few rules (but many exceptions due to the many influences). And it’s in use by large parts of the world.
On the other hand, Latin is beautiful and pure. It has more rules, but very few exceptions. You can also see how it influenced other languages (hello Jupyter notebooks…) But it’s simply not a very practical language today, as there are few people who use it.
I feel that choosing the language based on how close it is to thinking in maths is putting the cart behind the horse, because in all scientific and applied scientific code I've seen, the actual math models is something like 10-20% of the code, and the majority is all the data management and plumbing around that; so you should pick the language based on how easy and maintainable the majority of the code will be, which is not the 'mathy' part of it.
I beg to differ. If by "math" you mean arithmetic and calculus, then sure. Combinatorics, graph theory; probably most of discrete math is very much object-oriented.
Consider a graph G, with vertices (a,b,...,z). Say, we want to find the shortest path between vertices a and m. A mathematician might write an algorithm called shortest_path(G, a, m) which might call the algorithm breadth_first_search(a) etc. Functions are called on objects to determine their properties. Or to modify them into other objects. This is what you see in math or theoretical CS papers.
In python, in popular libraries, the above will be attained by something like
G.shortest_path(a,b)
which seems to imply as per OO that graph G has a property shortest_path. In mathematics we don't think like this. Because it prevents mathematical abstraction. An abstract algorithm, like say Eigenvalues can be applied to a matrix M or to a graph G. In a functional language, the user would just do Eigenvalue(M) or Eigenvalue(G) as needed.
In python, these would be M.eigenvalues() and G.eigenvalues(), which makes them distinct.
What you're saying is not true of the most popular graph package in Python (networkx). Graphs there are objects with attributes (edges, nodes, etc) and methods (add_edge, etc), but algorithms such as shortest paths are functions as you have called most natural: shortest_path(G, a, b)
It's still object-oriented, though. The graphs themselves are objects, which helps with ergonomics.
There are also functions that operate on graphs, because the alternative would be to stuff perhaps thousands of methods into the graph class -- which would be extremely annoying to maintain as well as abysmally slow.
Since you capitalized it, it looks like a static method on the “G” class. There is nothing OO about static methods - this is purely and organizational construct.
I think it's the superclass A of both G and M that defines the eigenvalue() function. Then when both G(A) and M(A) inherit from A, it captures the mathematical relationship. The subclasses indeed should implement the eigenvalues computations differently, like in real mathematics.
Funcional programming is used in Coq because it captures logical chains, for theorem PROOFS, but not for specific COMPUTATIONS, such as eigenvalues.
I understand that this is possible. But this is not how mathematicians think. If you got together a bunch of mathematicians and asked to invent a programming language to do mathematics, they would never ever invent the concept of a class.
Mathematics (outside of stuff like category theory etc) is simply sets and mappings between sets. Anything more complicated is forcing the user to think in a way that mathematics does not.
> But this is not how mathematicians think. If you got together a bunch of mathematicians and asked to invent a programming language to do mathematics, they would never ever invent the concept of a class.
Hi, mathematician here. Please read the source of SageMath: written by mathematicians, for mathematicians. Therein, you will find thousands of classes. Almost all computer algebra systems, written by mathematicians for mathematicians, have a notion of classes.
> Mathematics (outside of stuff like category theory etc) is simply sets and mappings between sets.
This is an extremely narrow-minded view of mathematics. Mathematicians thrive on abstraction, (category theory is literally mathematics; that's a very strange exception for you to carve out), and if we were to boil everything down to "simply sets and mappings between sets" then we'd be bogged down in utter tedium and nothing would ever get done. Object-oriented programming, specifically class hierarchies and inheritance, are extremely valuable for doing all sorts of math at high levels of abstraction.
Hell. If I were to be especially pedantic, I'd point out that category theory itself is "simply sets and mappings between sets" except that sets aren't quite large enough so it's actually "categories and mapping between categories".
Since the eigen decomposition is specifically a property of the matrix representation of the graph, but other things like shortest path distance between two nodes are naturally enough computed on graphs formed by objects with pointers, perhaps the correct interface is G.adjacency_matrix().eigenvalues().
There are a lot of hierarchies in the families of objects that discrete mathematicians are interested in. Inheritance is extremely natural in such settings.
I think the so much more/hard to even describe is sort of a problem?
There is a lot to it, and in my experience it's a very long road to getting your first problem solved.
There are a lot of very cool pieces but I want it to be something I can pull off the shelf once in a while and be productive rather than something that I need to be an expert in before I can get started.
Another big one that many languages treat as table stakes these days is the whole 'batteries included' thing. Whenever a new language comes out and I have to make my own (or install) core capabilities, it feels like a huge waste of time.
"A lot of batteries included" cannot be repeated enough. This is huge. Some languages you have to write your own. But the flip is also true. Take Node. For function X, there are 100 ill named "add ons" of variable quality and repute. Discoverability is a nightmare. Which-to-use in your mashup leads to paralysis analysis. I use a handful of libraries for some pretty large Python apps, but for so much of the basics, it's all in there. The quality and documentation vary a little, but the similarity/consistency is much better than you find in the wild. Even in Elixir (a much smaller universe than Node/JS), I experience this "15 poorly named packages that do the same thing in different ways".
Agreed. I am a Perl refugee. I was so tired of searching CPAN to find ten packages, but some wouldn't install on Windows, and then I would have to evaluate the survivors. They would cover sixty percent of the problem, maybe seventy, all different coverages.
Now people talk about Python's standard library being "where code goes to die," they want to remove old stuff, makes me sad.
I haven't found that in Elixir... I often find one of two that haven't been updated in a couple of years but still work just fine because the Elixir language itself doesn't change much.
It may depend on what types of libs you're looking for, of course. It's definitely missing a lot of stuff.
I am not a JS person so I dont know, but in the jungle of add ons I wonder why there isn't a project which provides a stdlib like package for JS. Is it because for JS you want to install minimal libs?
Having to piece together a Python installation can be an annoyance for a beginner. I always offer to help them set things up the first time. Since most of my workplace is on Windows, it's easy to have people install WinPython.
This is especially true if they've already tried setting things up, and have made a mess of it. Because WinPython is self contained, it can work on a computer that already has a working or non-working Python installation.
What do you mean by piecing together a python install? Most linux should have one pre-installed or easily available via the package manager. After that venvs and pip seem to be generally enough for most tasks.
Packaging Python applications that are outside of containerized apps can be a headache. You can't assume that the system's Python suits your needs, or even that Python versions in package repositories will, either.
A big pain point still remains when it comes to packaging and distribution.
It's not insurmountable, but if you're used to compiling and/or cross-compiling and shipping binaries, shipping Python apps can be headache inducing. The headaches increase when you're working with Windows or macOS, or using bleeding edge Python versions.
That said, for 99% of Python apps out there, the default tooling should be absolutely fine for packaging and distribution, as should 3rd party solutions like PyInstaller or Nuitka.
Oh of course, packaging python apps for final distribution is a different matter. I was assuming they were talking about the process of setting up a dev environment not deployment.
Oh yeah, 99.999% that's the case for dev environments. The only issue I've seen is that some versions of Linux distros have Python shipping with Tk and cryptography support in separate packages to be installed from repositories, where they might be installed by default on other distros, or are just included with the Python package.
I've been using Python for about 6 years, my last company wrote it's core commercial product exclusively in python. I've never used venvs, but I've yet to run into a situation where virtualenv (maybe with virtualwrappers as sugar), and pip + requirements.txt haven't been able to handle even fairly complex install situations.
My workplace is a Windows shop. I'm multi platform, but by and large the folks using Linux don't need my help. ;-) So when I help people its almost always on Windows.
I think the desire for batteries to be included is a holdover from the time of really really bad (or no) package managers. With a good package manager it doesn't matter if your batteries come included or not. It also provides a way for packages to battle it out external to the language -- compare Python’s `urllib` and `requests`.
The curation of the batteries is the value add. Which grpc package do I want? Which json? Which random number generator? Finding which rust crate is the norm for any given thing, is a task in itself. Then there is the security aspect. Pulling in hundreds of repos with many authors each, is a security risk.
The curation you're relying on is probably not delivering what you hope. If you picked a Rust crate by looking at a SO answer from say 2020, you're probably getting a similar quality of "curation" in Python's standard libraries.
Not because the Python maintainers are intentionally doing a bad job, it's just that their mission isn't to substitute for your research. To them, "Does last year's Python script still work?" is always higher priority than "What's the best way to do it today?".
And what Python's security team decide constitutes a reasonable security fix may conflict pretty badly with what you need.
"Does last year's code still work" is very important to almost all programmers. I wouldn't want to accidentally use a library that gets abandoned or deprecated a few months after I made it an integral part of my software. Being able to mostly stick to the standard lib makes this a lot easier.
That works for Rust's standard library, which won't abandon anything stable (stuff gets deprecated, but that just results in a build warning).
It doesn't make as much sense for Python's standard library where you'd need to be paying attention to goings-on in the community to know if one of the batteries that are included has been judged expired and will be removed.
SO would disable a question of this type for being opinion-based.
I would assume the Python devs review the code for a package before including it in the standard lib, right? That seems like a pretty big deal, well beyond someone just telling you that they tried a package and it worked for them.
Exactly. The single biggest hurdle for me using some open source tools written with newer languages is the ridiculous dependency surface due to a preference for package repositories over curated “batteries included” bundled libraries. I work in an environment that is hyper sensitive to supply chain integrity, which means a lot of languages that have a culture of “use all the third party packages” are out for us.
Give me a batteries included standard library please - much easier to audit and track.
Nearly every place I've worked, the supply chain attack problem has been solved or at least mitigated by using an Artifactory instance with a curated list of packages.
Perhaps the next programming language / ecosystem can remove this problem entirely . Why is it so hard to have access to everything while shipping only what is actually needed?
The need to runtimes shared by multiple applications on a single machine makes this a hard problem to solve, unless it's statically compiled. Of course, Docker renders this moot, and I'm unaware of any languages written in a "Docker-first" manner.
> On the other hand, some features are simply over the heads of beginners, making the code found in packages practically unreadable.
That's too bad. I taught myself Python last year but have kept it pretty "101". That is, I write code simply. Since I am new to Python I don't know if the unreadable code you're describing is because it needs to be (perhaps I am still tooling around in the parking lot of Python) or if it is the result of programmers being too cool for school (ha ha).
Perhaps if the latter I ought to blame the language for giving programmers a basement of esoterica to plunder rather than the programmers themselves.
As obtuse as C could get with pointer arithmetic and the cryptic looking "question-mark-operator", there really wasn't much esoteric stuff in C. Contrast that to my experience with Swift where seemingly every week or so I would come across cryptic code where someone was using a capital T or some bizarre case expression I am unable to mentally parse.
> Turbo Pascal
You suddenly made me miss an ex-girlfriend. Maybe they still code in Turbo Pascal in Portlandia.
There's a fine one just like the ex, Free Pascal, waiting at https://wiki.freepascal.org/ to be checked out. Now the dream can become reality, but do take into account that satisfaction can depend on being realistic.
I think one big factor is easy entry but almost no limit:
Like BASIC back in the day it's a language that allows newcomers to do things long before they are comfortable with the added syntax requirements of C based languages
Despite that the sky is the limit, almost all problems a developer will encounter can be tackled with python. Yes for many problems there are languages that are better suited for the task, but in most cases a project does never even reach a state where that will matter.
And it's installed by default in many Linux distributions.
But I think this may be evidence for the argument that scientists should be paired with research software engineers who take the science codes and turn them into usuable libraries. Scientists don't necessarily need to produce codes that others use, simply ideas that can be translated into useful libraries. Certainly some scientists can do this themselves, but not all can nor should be expected to.
Some projects are distributed on PyPI as source code, and sometimes that source code has C modules that need to be compiled. The same can be true with packages that ship with Rust modules.
Both might require you to have compiler toolchains installed.
It's rare, but I've run into packages that don't install easily with pip, due to not having binaries for a specific Python version + platform (something like this one where the combinations of Python+platform+MacOS version are limited, for example - https://pypi.org/project/virgil-crypto-lib/#files ).
Also there are probably tons of "packages" which were never published officially to pypi but exist on github, that people have made public but decided not to maintain.
Similar camp. Two reasons I enjoy Python, both related to documentation:
1. docstrings
2. keyword arguments
Docstrings mean I can do REPL experiments without the additional friction of opening a browser to RTFM. Keyword arguments reduce how much I have to lookup the semantics of a function's signature.
Some of my reasons for choosing Python in the first place have been mooted. Python is free, and multi-platform, but so is everything else nowadays. I think that battle is over.
For my use, packaging has not been a big obstacle, maybe because I tend to use a relatively small handful of established packages. I don't remember needing a package that I couldn't install with pip.
Easy to learn? Yes and no. I think a beginner can ease their way into Python and get productive, if they start with basic scripting and progressively adding a few language features. On the other hand, some features are simply over the heads of beginners, making the code found in packages practically unreadable. But I've never found any language to be better in this regard.
Fast? Sure, with numpy in its domain of use. Or writing your own C libraries. However, I think a beginner using numpy will run into roadblocks that require help from an experienced programmer, who can visualize what the array operations are doing under the hood. So, writing fast code isn't easier to learn in Python than in any other language.
The editor should have a final option, that turns your code brown if it's just so bad that nobody should ever have to read it, including yourself.