When I first tried to learn Python last year, the frustration of being a first-timer dealing with the Python 2 vs. Python 3 split very nearly turned me off of the language altogether. While I loved the language itself from first sight, Virtualenv was the tool that finally made Python a joy to use, rather than just a joy to write. This is a very good move.
So, as someone just dipping their toes into Python, what did you do regarding 2 vs 3? I have been tinkering with 2 but should I go wholly with 3? Help me before I just go back to Perl 5... :)
Learn Python 2 first, since it's vastly more supported right now, but know that it's very easy to jump between Python 2 and Python 3, and many people do. The decision of which to use on any particular project is basically:
1. Are you doing web development? If so, use Python 2.
2. If not, do you need or want to use PyPy? If so, use Python 2.
3. If not, are the libraries you want available for Python 3? If so, use Python 3.
4. Otherwise, use Python 2.
You know, I hate to say it, but you, and your parent commenters have basically said quite a bit without actually answering the question. Anyone coming here wondering which one to learn is going to be left with the same question, only slightly more confused.
I purposefully avoided any attempt to answer the question directly.
How would one know what to recommend while knowing nothing about `jcurbo`s intentions/designs/requirements?
The stackoverflow and wikipedia communities sometimes get flack about maintaining strict objectivity, and I know the HN community is allowed to be different, but surely you agree that there is no right answer to `jcurbo`s question. And if there is no right answer, surely offering information from a trusted source can be just as helpful (if not moreso).
> How would one know what to recommend while knowing nothing about `jcurbo`s intentions/designs/requirements?
For someone learning Python? Just give them your best suggestion. Look, it's fairly easy.
Learn Python 3. There are differences, and you might run into issues later on, but once you get to that point, you'll know enough to do something about it.
See, when someone asks whether to learn Python 2 or Python 3, that's what they are asking for. If they are learning for specific reasons (for a job, lets say), then instead of trying to answer, you ask follow up questions. But for learning a language, they are seeking advice from experienced people.
So, when those same experienced people, who already know Python 2 and/or Python 3 tell them to learn Python 2 and/or Python 3, they are being told: we don't know.
And how is someone inexperienced supposed to know which one to learn when experienced people don't know?
Basically, if you can't answer the question because you feel you lack information, ask. Otherwise, answering the question "Learn Python 2 or 3?" with the answer "Learn Python 2 or 3." isn't helpful.
Thanks to you and everyone else that responded. I was more interested in the parent's actual experience, what made them frustrated, etc. I agree that it depends highly on personal use cases and what you are trying to do. Personally, I am using it to support some of my graduate studies and some personal projects, nothing major. If I was doing major software development I would be doing more research on my own, and indeed this line from the wiki page would seal the deal for me:
"At the time of writing (July 4, 2010), the final 2.7 release is out, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that."
Since what I am working on doesn't involve major libraries (like numpy) I see no reason to stay on 2.x and I think I'm safe to look at 3.x for future stuff.
It almost always is. The term has a connotation of "the existing state of affairs for the majority of people". So the majority of the web is largely template driven web pages/apps, served from php or java or maybe python behind apache, with some stuff enhanced by javascript. That is the status quo. All of us here of course know that is largely going to go away (and it's actively even happeninge), replaced by javscript driven single page apps where templating happens client side, and the web server becomes a file transfer mechanism for the code and some data, a well tested way of doing rpc for the rest of the data.
So yes, there are pages doing this new style web right now, but that doesn't change that the mojority of web stuff is the older style. So yeah, the current state of affairs is that the web is mostly outdated.
This is necessarily true always, unless that which is being examined is stagnant.
So, as someone just dipping their toes into Python, what did you do regarding 2 vs 3?
As someone who was learning Python for fun personal projects rather than serious business projects, I decided to use Python 3 wherever possible, while familiarizing myself with Python 2.7 just in case. It turns out the two versions really aren't all that different. A lot of people will still recommend starting with Python 2.7, since it really is such a small jump from there to using Py3k; I essentially did the opposite. Both approaches work. Finding tooling for Python 3 might still be a challenge (I remember having to fix one or two bugs in IPython before it would run at all), and the usefulness of these tools to a beginner is really the biggest reason why you might want to stick with 2.7 for the moment. But if you're adventurous, I do highly recommend Py3k. Especially with the upcoming 3.3 release (due later this year), I'd argue that Python 3 finally has enough new features (and enough library support) to justify using it full-time.
(When will it be possible to know, beyond doubt, that Python 3 is ready for prime time? When third-party implementations like PyPy and Jython make the jump. I don't know about Jython (which just recently made it to 2.7), but PyPy's working on it as we speak.)
Unless you need a library that's only available in 2, I'd use 3 at this point. But virtualenv takes the pain away from having to choose, since you can just create a new "environment" with 2 or 3 depending upon your needs for a project.
If you're coming from the Ruby scene, think of virtualenv as Python's take on RVM.
As a beginner, picking between 2 or 3 should not be one of your concerns, it's a choice that does not unlock much value either way (and it does not hurt much if you pick the 'wrong' way either).
I would say start on 2.7, as most of the popular Python libraries are still Python 2.x only, and 2.7 backports some Python 3 features, so you (and your code) will be ready for 3 whenever the world switches over :)
Python 3 was on a 5-year transition. This was to let the tools develop, give time for major libraries to be ported, etc. We are now more than halfway through, so most of the important libs work fine in Py3. And the syntax is generally nicer, so you might as well start with Py3 today.
Zed Shaw originally strongly advised students in his Learn Python the Hard Way course to go with Python 2, because there's less opportunity for confusion (eg, if you start with 3 there's a good chance you'll come across libs you need that haven't been ported to it yet).
However, I can't find that warning anymore in the second edition, so not sure if he's changed his mind or not.
Learn Python 3, it has nice new language features and some of the hairy parts of older versions have been replaced with new and consistent constructs.
If you need to ship and launch a software that absolutely depends on libraries that only work with Python 2, it should be rather easy to not use the newer features and go with Python 2.
Awesome! I'm hard pressed to think of a third party tool that makes more sense for core-inclusion than virtualenv. Hoping the virtualenvwrapper stuff will also be included (or at least similar commands will be)
I'm personally not fond of the virtualenvwrapper porcelain: cd foo; workon foo is an extremely marginal improvement over sourcing the activate file, especially when routinely opening and closing terminal tabs and windows.
I use a number of zsh functions and hooks as porcelain to virtualenv so that it behaves like oh-my-zsh bundler support [0] (which I've shamelessly ripped of and tweaked too) i.e being inside foo directory is sufficient. This works especially well since new tabs open straight into the current tab's wd.
I don't use virtualenvwrapper either. I generally have more stuff to initialise for my projects than just the virtualenv. I use swork[1] a tool I wrote for managing your shell environment. It basically dumps all of your environment variables to a file allowing them to be restored later, then it runs a project specific initialisation script.
You might want to consider using the virtualenvwrapper.project extension. You define a "projects" dir (which I had already) and then it binds virtualenvs to projects so `workon foo` automatically `cd`s you to the right place. You can also create a new project dir and virtualenv in one shot with `mkproject`.
Even though I agree, virtualenvwrapper is currently a shell script. That makes me think it probably wouldn't be included, but a pythonic version of the same thing could exist and fill this gap.
Also a fan of virtualenvwrapper. workon with tab completion (in bash, at least) is so nice when you've got more than a few virtualenvs and can't quite remember your naming scheme :)
Does anybody else think that the necessity of virtual environments for programming languages and virtual machines for applications is a sign of failure in package management and sandboxing in modern OSes?
Sort of, but they necessarily approach the problem from different vectors.
An operating system design with a seamless, straightforward, easy-to-use mechanism for handling all potential dependency conflicts on a multi-purpose, multi-user system is a very hard problem. Any solution is likely to be complex and potentially confusing to users, such as debian alternatives or the modules environment system. Every language has its own library management system: cpan, pip, rubygems, leiningen, etc., and they all profit off the assumption that they are managing a single set of requirements and dependencies. Operating Systems do not have that luxury.
On the other hand from the user perspective, they may not even have root on a system and as such may not even be able to install libraries without contacting a system administrator. Maybe it's a shared hosting environment, maybe it's a research compute cluster, maybe there are SOX restrictions on developers, whatever. Virtual environments focus directly on giving the user/app what they need and isolating that user from everything else.
100 different users can use virtualenv to create 100 different, completely not-compatible python environments and they're all happy. How would an Operating System do that, without essentially duplicating what virtualenv already does? (Not saying that OS developers aren't going to try, which Zed Shaw will then ridicule, but that's another story). For sandboxing in general, at what point do you create the sandbox? Usually for each individual app and usedr you don't want to have to answer questions about whether you still want the C++ compiler available or whether a virtualized /etc/passwd is needed. Those are the sort of questions you have to answer when designing OS-level sandboxes.
>> How would an Operating System do that, without essentially duplicating what virtualenv already does?
This, I think, is backwards: virtualenv must do it already because the os doesn't. I concede that creating something now may be too late not to suffer the "15 competing standards" effect.
You can always install from source and pass --prefix and --exec-prefix options to ./configure. Virtualenv is not strictly necessary to run in an isolated environment.
And as for whether virtualenv should have been part of the OS... it already is in some cases, to the extent that any userland tool is part of the OS.
$ uname -o
GNU/Linux
$ which virtualenv
/usr/bin/virtualenv
Does it really matter whether it was the python developers who got there first?
This. Python's eggs and repos are a far cry from proper software repositories, such as ones available through apt. Tools like pip do not keep track of what eggs are installed, which ones depend on others, etc. It is up to you to keep a requirements file. If you have more than one project that need to share an environment, now you have to find a way to merge two requirements files. If you ever drop a dependency on a package, it's up to you to manually uninstall it and its dependencies.
Don't get me wrong, pip is great for what it does. It just is not anywhere near what we should find acceptable because of the things it does not do.
Virtual env installs libraries for your program/web service/etc.
Those libraries will eventually need security updates.
On Linux for example, which hosts most servers, you upgrade python libraries with yum update/apt-get update/you-name-it.
Single central package database, security updates are marked as security, makes upgrades easy and you never miss one.
With VirtualEnv your dependencies are installed outside the score of the package manager. It means you have to keep track of security updates yourself.
If you don't, and one lib has a well known security bug, you'll be hit and you most likely had no chance to update because "you didn't know".
That's why I'm asking. I know a lot of companies hosting python things and using VirtualEnv because "its easy", but they entirely discard updates. I'm not sure that is wise. I was hoping someone had a solution.
> On Linux for example, which hosts most servers, you upgrade python libraries with yum update/apt-get update/you-name-it.
Not really. In many cases distro provided packages are unsuitable, either because they are just outdated, or because they have some strange patches merged in. So people are already managing their python packages with pip, virtualenv makes it easier.
If you want to use distro packages, then virtualenv allows you to use those (it does that by default). You only need to install into virtualenv those libraries that are unavailable/unusable from distro packages.
> I know a lot of companies hosting python things and using VirtualEnv because "its easy", but they entirely discard updates.
Discarding updates (without due care) is bad sysadmining regardless of what kind of package management is in use. Virtualenv is no magic bullet, nor is it an excuse for sloppy procedures.
So what you're proposing it tracking security updates manually and update your libraries?
As much as it's hackish, it's possible to update virtualenv's libs. But tracking the security update is another story.
If not tracking hundred of libs manually is bad sysadmining, well, I'd like you to be my sysadmin and track them.
There's even a bigger issue here: many of those libs have zero tracking. That means even if their author fixes a security bug in the source, he probably won't mark it.
And you, not being a sloppy sysadmin, how will you deal with that? It's pretty obvious to me that you just don't deal with it.
Then one day one of those libs will be widely used, have a bad bug and everyone will scour their systems to update.. but of course the damage will already have been done.
It's pretty common to deploy Python apps via a virtualenv. In most cases you will use something like a pip requirements.txt or fabfile for Fabric to define how to deploy your app and what are the dependencies. The file is tracked in version control along with the rest of your project.
If at some point you need to upgrade one of the libraries due to a security vulnerability, you would change the dependency in the requirements file and roll out a new version of your app to all your servers.
If you have multiple apps, then repeat the process for each app.
Generally it would be a bad idea to rely on a single installed version because often times the API can change between versions and different apps may target different versions of a library. It's preferable that each one has its own dependencies isolated in a virtualenv unless you have a really rigorous process for keeping all your apps updated to the latest library versions.
Your parenthetical is just wrong. The setup for a new virtualenv requires access to the internet, iff you choose not to a) bundle the packages in a libs dir or b) have a locally hosted mirror of pypi or c) have the libs in your system packages already or d) a dozen other things.
Of course this is true for setting up any development environment. You have to get the dev environment from somewhere at some point to bootstrap the process.
Your entire parentheical screams FUD, and obviously disingenuous FUD at that.
That's good news. Last I looked at virtualenv, though, it could have used more oomph. At the time (last Fall), I was jumping back and forth between Python virtualenv environments and Ruby rvms. RVM was a bit more of a pleasure to use. I hope the virtualenv developers have closed the usability/feature gap since then.
Virtualenv is a nice tool for managing library sets.
RVM allows you to do that plus it allows you to install and manage multiple versions of Ruby itself. Want to have side-by-side Ruby environments of 1.8.6 and 1.9.3? No problem.
Interesting, I've had an almost opposite experience, but I don't know the ruby landscape all that well (bundler/gemsets/rbenv/rvm etc) and I suppose it just depends on what background you're coming from.
FWIW, when creating a virtualenv the -p flag lets you choose a python binary (and therefore version).
rvm will download the requested version's source, apply any patches or flags you specify and then compile it. You don't have to do any work other than installing rvm. The rubies are also stored in ~/.rvm so your system doesn't get cluttered up.
Rvm also integrates into your shell so that when you cd into a project directory with a .rvmrc file, it will source the file and load the proper ruby and gemset for that project.
Virtualenv is much closer to ruby's bundler gem than rvm.
Second on that. I find that pythonbrew fits the way I actually work with multiple installs much better than virtualenv (even with virtualenvwrapper) does.
I only skimmed PEP 405, but did anybody get a sense for how this would work with regard to managing different versions of Python per virtualenv? It seems like a tool to do that necessarily has to be higher in the stack than the Python interpreter.
I guess the same way as with multiple python interpreters. E.g. under Linux/Mac OS X I might have python python2.5 python2.6 python2.7 and etc. Under Windows I have different locations for different python versions (e.g. c:\python26 c:\python27). You will simply have virtualenv, virtualenv2.6, virtualenv2.7.
Here's the link to PEP 405:
http://www.python.org/dev/peps/pep-0405/