The minor improvements of Python 3 did not warrant breaking backwards compatibility and most could have been handled in a way that would not break it opt-in directives.
The very swarms of users that chanted “just upgrade” as if that not incur a significant cost also seemed ridiculously naïve to me, not understanding the real cost that large projects have by having to rewrite very extensive codebases and dealing the potential regressions that that might involve.
Everything about the switch, from it's very conception, to it's execution, was handled in a veritably disastrous way by the team, that really did not seem to appreciate even a fraction of what obviously is involved with projects that have millions of lines of code and of course would rather not have to rewrite that all.
This is why many projects such as Linux, Windows, Rust, Cobol, Fortran, C, and C++ take backwards compatibility quite serious. Serious enterprises do not like to invest in something if it mean that 10 years later they would have to rewrite their entire codebase again.
Even on my home computer, I simply do not have the time to rewrite the many Python 2 scripts that have written over the years that run my computer. — it is cumbersome enough that once in a while part of my desktop stops functioning because my distribution removed a Python 2 library which I had relied upon as a system library that I now have to install as a user library but that was hitherto quite easily fixed.
Do these men think that time is free?
Rust's commitment to backward compatibility is certainly extremely commendable, but I don't think the language went through anything resembling the switch from Python 2 to Python 3 in terms of breakage.
Some of the changes in Python 3 are very fundamental. Imagine if Rust had shipped without String/str and they were added after the fact, would Rust manage to avoid splitting the ecosystem? That's an open question as far as I'm concerned.
And I also hope that we never find out. Rust's fundamentals have proven to be very solid so far. Having things like OsStrings (something missing from most programming languages, including Python 3 AFAIK) shows a great amount of foresight and understanding of the problem space. Contrast that with Go which seems very intent on completely ignoring 30 years of programming language evolution.
Can anyone tell me why statements such as these get the emotions going?
There will come the day when something will change semantics, or require different kinds of runtime support across editions, and then the headaches how to link binary crates from different editions will start.
Editions to me appear only to work, provided everything is compiled from source code with the same compiler, aware of all editions that came into use.
Python is one of the most "accessible" languages in terms of the actually programming experience, but making a project reproducible is a nightmare. There doesn't seem to be a real "right way" to manage dependencies, and getting a project running seems to often start with figuring out how the author decided to encapsulate or virtualize the environment their project is running in, since changing your system python for one project can break another.
I know it's an older language, so many lessons have been learned, but after working with Rust, or even NPM it seems amazing that developers tolerate this situation.
Some combination of requirements.txt (which lets you dial in, with great precision, each of the libraries you need, and is trivially created in < 50msec with "pip freeze")
1. Containers - That's it. You control everything in it. 2. virtualenv - Every environment comes their own version of python and their own set of packages and versions of those packages. Add virtualenvwrappers and you can create/switch trivially between them.
It's been at least 2 years since I've run into a issue with python and dependencies that wasn't solved by both of those approaches.
Pipenv could have been it but it never got far.
Perl and Python are the only two examples of this to my knowledge: most open source languages do fine introducing breaking changes in major versions.
The question is why Perl and Python had such problems while for example NodeJS, PHP (comparable webserver scripting languages) have had no such issues.
I wonder is it anything to do with the areas they're used in (Python & Perl are popular local/cli scripting languages in addition to web—has Bash had similar version woes?), or is it purely that the changes they made were more significantly breaking than others'? That's probably true of Perl6/Raku at least.
So in many of these cases the end user doesn't have a choice to use Python 3 until it's on offer.
And the vendor has usually integrated Python at a binary level into C code; that's why they provide a Python API.
The answer could even be "Red Hat Enterprise Linux 6"; consider that Python 2 is the default in this OS, which ended official support only at the end of last year. Many enterprises _chose_ this platform for its longevity, along with 3rd party vendors of commercial software.
This won't change that at all; pip/pip3 is a distro packaging thing, and any distro that packages legacy python2 as “python" and Python 3 as “python3” will probably continue packaging legacy pip-for-python2 as “pip” and pip-for-python3 as “pip3”.
IIRC the language known as "Perl", version 5, when it gets to the point when it is going to update it's major version, will skip 6 and go right to Perl 7
But there was a sort of broken promise given by the Python creators: Python3 was almost like Python2, but every library author had to review and repackage their libraries, anyway.
At that point, Python 3 should have been unambiguously incompatible with Python 2 :
- the only allowed file extension should be py3
- all environment variables should have been duplicated with a "3" (it shouldn't read or modify Python 2 env vars)
- all installation folders should have been duplicated with "3"
- all tools like pip should be suffixed with "3"
- and most importantly, it shouldn't try to optimistically run previous Python2 code or previous v2 tools
Still today, you can run "python" in a recent version of Ubuntu or Fedora, and it will be Python 3. Only "python3" should be possible. Distro are repeating the same mistake than with Python 2, and we will struggle again with Python 4, if there is any Python 4.
Many headaches wouldn't have happened if "python" was reserved to Python 2.
Pro tip to language and distro maintainers : make the major version part of the language name and executable, from version 1.
That would have made cross-version codebases impossible, and that's what ultimately allowed migrating. One-shot migrations were not convenient, or successful, or even effectively feasible for complex enough projects.
What allowed the migration was community experiment in cross-version sources, as well as reintroduction of "compatibility" features into Python 3.
In your shell you don't run "java8" or "java11", you just run "java", and then it's the matter of what version of java JDK you have in your PATH. The same with all other language interpreters and compilers, you don't run gcc9 or node14. Why doing something different for python?
Really, the mistake of python3 was to break compatibility with past programs. A lot of changes could have been done more gradually, in the first version require a __future__ import, then gradually remove compatibility with old features, and then remove them completely, making the new way the default and thus no longer require the __future__ import. And I think it will be the way for next python version, so in theory we will never have the same problem again.
Also to me it was an error for distros to continue packaging python2 as python. Other distributions, like ArchLinux, switched everything to python3 as default a lot of time ago, it's only Ubuntu that continued to ship python2 as python, thus making a lot of programmers relying on it. It would make sense that the command without the name refers to the latest version, and not the legacy one.
That's not the case with C# or Java.
I think Python's failure is unique. It needlessly broke too much back-compat at once, provided too little benefits to make up for it, and let everyone drag their feet for a decade with the upgrade.
"We have nearly endless money and therefore we can make any new feature by treating older versions of our langage as bytecode and write transpilers for it".
However, I do not disagree. Renaming is the right thing. A version number is easily omitted.
Compare this to Python 2/3. It's basically an incompatible fork that doesn't add enough for many projects to consider upgrades, and adds the overhead of having to worry about two version. All it really accomplished was guarantee that "Python 4" will never, EVER be a thing.
The only distro I'm aware of which tries to protect users shooting themselves in the foot with pip is Gentoo. Most of the others will happily let you "sudo pip install" stuff and lead people to think that's the correct way to do things.
Unfortunately pipx has come too late. Pipx provides a real way for users to install arbitrary python tools, but too many docs out there tell users to use pip. Then you've got all the users who want to "play around" with python and install libraries. Even things like jupyter have crap support for virtualenvs and make it far too easy for users to have all their projects in a single env. It's a mess.
Regular users should never have been exposed to pip2/pip3. They should never have even been interacting with the OS python interpreter. Pip should only exist in a "global" contexy to support bootstrapping virtualenvs and nothing else. Poetry does a lot of this right.
Not sure what any of this has to do with open source languages, mind clarifying?
if you step back and think about it, using the same word for two fundamentally irreconcilable things is mad.
I give their decision a week before they revert and come up with an alternative.
Also they aren't withdrawing pip from being available for python 2. You just use the older version of pip. Programs that need pips newer features won't install on python 2 anyway. Because it's really going away very quickly now.
Perl 7 finally is going to break this compatibility, at least according to the current plan, but the "porting" still is supposed to be trivial, in most cases just adding a pragma or two. 27 years after Perl 5 was released...
This is a good example of what I was talking about. Python 2.x isn’t “going away very quickly now.”
(AIR isn't an interesting solution for a big chunk of apps, but there's lots where moving out of the browser is fine)
But the Python communities I've interacted with seem to care deeply about the ecosystem, updated libraries, new features. Web apps, data science, etc. Is my experience here warped? Are there people here seeing Python get used for the kind of stuff that might live as long as Cobol without moving forward with the language?
A tool is a tool. When I bought a new drill I used it to put together the playhouse for my kids rather than tearing down my fence just so I could put it back together.
More likely, enterprise vendors like Red Hat will maintain Python 2 + pip for a long time.
The upstream project has no reason to do so and made the right decision to stop supporting legacy stuff.
There is also a straightforward way of "compiling" your code - install all your dependencies into a virtualenv, tar up that virtualenv, and untar it to the same path on the next system. I've seen this system used to work around pip and Python upgrade complexities that already exist, even within Python 2 alone, and it works fine. Then you're insulated from both changes to pip and changes to the actual packages you're trying to install from pip.
Edit to answer your question because someone downvoted me and now HN is preventing me from replying: I don't think you'll need to do anything special. The common case, honestly, is that you already have some version of pip shipped with your system (e.g., you're using Python and pip from your OS) - just keep using that version, because by definition it's a Python 2-compatible pip. The official Python 2 binaries bundle a version of pip which is also by definition Python 2-compatible.
If you need to upgrade pip for some reason, you might need to specify a version constraint on it, e.g.
pip install -U 'pip<21'
pip and the legacy easy_install both access the "simple" list for a package when determining download options. This is a basic HTML page with links to all public versions of a package. Here's pip's: https://pypi.org/simple/pip/
These links contain Python version specifier information, which pip can use to select an appropriate version. For pip 21.0, that specifier is ">=3.6", so any modern pip will know it can't be used on a Python prior to 3.6. It will therefore fall back to the nearest version that provides a compatible specifier.
Looks like this was implemented starting in Pip 9.0 (at the end of 2016). From experience (my product is written in Python), there are plenty of enterprise installs that still use much older versions of pip than this. Those will grab pip 21.0 or newer (I just confirmed with a copy of pip 8).
I know for us, that'll be important to document. We still support Python 2.7 in part due to slow-moving enterprise installs, and I'm sure we're not alone in that. So geofft's example for forcing installs to <21 is exactly what a not-insignificant number of people will ultimately need to do.
But I don't think that they will. Who is going to make them?
This is a very conservative hunk of software because this is not a "move fast and break things" industry. This is a "people die when we screw up" industry. So they haven't moved from ArcGIS 10.2.1 for our version. Change is coming but they have to be careful.
So ArcGIS 10.2.1 comes with a python install of 2.7.2 (I am pretty sure it is .2). And you are very strongly advised NOT TO UPGRADE THAT in big bold letters on the relevant help pages. So I need to use 2.7.2 to manipulate some stuff out of ArcGIS 10.2.1 to do "stuff."
And there I sit.
ArcGIS 10.2.1 comes with 2.7.5, [1] and is very, very old. It was released in Jan 2014 and hit EOL in 2019. [2]
All not-EOL versions of ArcGIS Server support python 3.x. But if you're using ArcMap, upgrading to a later version will not move you to python 3.x: you'll have to migrate to ArcGIS Pro for that. (note for the unaware: this is not a "well just buy the more expensive thing" post. ArcMap and ArcGIS Pro share the same license.) I recommend migrating to Pro for this and other reasons, but I understand that a lot of orgs don't have that on their roadmap.
Out of curiosity, is there a technical reason why your org isn't using a supported version, or is it an "ain't broke" situation? I'd ask migrating to ArcMap to Pro, but if you're on 10.2.1... yeah.
[1] https://support.esri.com/en/technical-article/000013224
[2] https://support.esri.com/en/Products/Desktop/arcgis-desktop/...
So I assume that the geodatabases, the address locators, and such are all the sticking points in terms of the "products" of ArcMap. I suspect that sometimes their software dips into that specific version of arcpy, as well.
Until the vendor hops, we cannot hop. It's a problem that will be eventually fixed, but I bring it up because many people here at HN think of greenfield development often, and also in industries where tons of iteration are met with approval.
I can understand that in some industries you don’t easily upgrade, because of testing and certifications. It’s however also possible to put of upgrades for so long that it becomes reckless. If you’re stuck on Python 2.7 in 2021, and your code could kill people, you’re putting people in danger by being reckless.
I’ve also seen people claiming a need for stable platform as a reason to not upgrade. Well, running Windows Server 2008 in 2021 you indeed have a “stable” platform, but it should come as no surprise when modern protocols and encryption isn’t working.
It's comments like this that make me very sad for the software industry as a whole.
No, he's not being reckless and he's not putting people in danger. The people who make breaking changes to languages, APIs, operating systems and libraries are the ones that put people in danger and that cause others to stop upgrading because they quite reasonably fear breaking changes that could have far reaching consequences.
The price of backwards incompatibility is poorly understood.
The customer was running Red-Hat Enterprise 5.0, and it was going to stay that way as long as Red-Hat would be willing to support it.
So it’s not like it has been feasible for most projects to migrate for a full decade, it’s only been a few years.
If you haven't already, I'd recommend setting up a PyPI proxy and backing it up. I don't know what PyPI's policy is around supporting old packages or versions of the package manager are, but I have learned the hard way from working on other legacy software stacks that things have a way of disappearing from the Internet when you're not looking.
So if it isn't in the standard library, I haven't been using it. Pity, because lxml seems like it would solve some of my issues.
So the first thing I would have to do with such Python is monkeypatch ssl.
Sadly, I have no idea how to monkeypatch SSL.
There you chose to sit.
If 13 years isn't enough warning to migrate your use case, no amount of warning will help you.
At some point you have to stop saying its to hard to migrate.
“ It's kind of disappointing that developers aren't self-aware enough to understand that they are essentially screwing themselves by not moving quickly in dropping "legacy" support.
"Python 3 will be available in 2008 but we understand that you won't use it for at least 5 years."
There's an entire class of developers who won't upgrade until they absolutely have to. There's also a class of administrator that won't upgrade their current PC browser from IE6-IE8 until they absolutely have to. Developers are basically screwing themselves by not drawing a firm line.”
One option could have been to spread the brraking changes over multiple versions, but given the bad state of version/dependency management in Python, this would have likely been a clusterfuck too.
Best option would have probably been to have a longer period of RCs and only release 3.0 with the performance regressions fixed. The myths around the slowness of 3.x (especially for scientific libraries) stuck around for a very long time after they were fixed.
I suspect many python 2 shops will maintain the old and move to something else for the new. Maybe the "new" thing will be Python 3, but I expect this will give other languages opportunity, because people do have emotions, rational or not.
Hell, numpy 1.20.x is dropping support for anything below 3.7!
I started doing science python in 2015 on 3.5.x, never had to touch 2.x at all.
The definition of insanity is doing the same thing over and over again and expecting a different result. (c)
My advice would be to gradually port what you can using six (https://pypi.org/project/six/)
[1] Fine in the sense that, the code will still work
Screwed? No. Terribly inconvenienced, yes.
This is assuming you have enough other stuff to do that you cant afford the “debt” of migrating the python2
there has been no reason now for years to start any new project in python2, and since it went EOL last year you would have to be crazy to use it.
Oracle are doing that with Java too.
Quickly made a lot of our clients untick the IE6 requirement from the wishlist.
This is where you are quite wrong.
Many languages and other software projects take backwards compatibility quite seriously.
Backwards compatibility is far more interesting to many clientele than the quality of the language itself. — the idea that one would have to rewrite one's codebase again later with all the potential regressions that that might lead to is not an appealing prospect.
Planes would crash if the software that ran airports would have to do this.
Packaging Python 2 & 3 together, like gcc, would have had another big advantage -- it would have meant early on I could assume everyone had Pythkn 3. There are still, plenty of macs with 2 but not 3 by default.
On the C++ side, C++ didn't get more spec until C++11 (2011), then it took a decade to be supported by compilers. Imagine how long it takes to roll out a language/library change when compilers themselves are on a 3+ years release schedule.
Either way, the C++ world is heavy on proprietary extensions and tools. There is little care about standards. The standard itself is hard to consider a standard, all it does is suggest tens of new features -some of which are impossible to implement- that may be added one by one over the decade.
You probably could run GraalPython, which is Python 3 and Jython, which is 2.7, since they both run inside a JVM.[1]
There'd be no interop between the languages, though. To get that in Graal, you'd have to reimplement python 2 using Truffle.
That's a massive effort compared to simply running 2to3 to migrate your old code to Python 3.
[1]: https://www.graalvm.org/reference-manual/python/Jython/
Until recently it was easier to set up for Python dev on Windows because you could download a clean Python and not worry about broken Pythons on your path.
Microsoft wrecked it all last year by including a system Python and they'll probably play games to keep it in front of your Python on the path.so you will be typing pip3.7, pip3.8, pip-3.6-pypy, etc. for the rest of your life.
Yet no matter how much I try to educate users some of them still feel entitled to use the system Python and I find it disheartening to try to suppport those people.
I have found that my PATH entry reverted at least once after I tried to hide the system Python and I am also wonder if Microsoft will use their 'monkey patch the standard library' approach to forclose any stopping the insanity.
Poetry has been solid, at least lately.
If Pip dropping py2 support today kills your app, it wasn't fragile, you were negligent.
I don’t have any rights to python. If I chose not to rewrite my code for python3, I am increasing the fragility of the world.
However the same is true for the python maintainers. If the maintainers were interested in maximum anti-fragility, Python3 should have been a fork.
I’m not making any sort of moral judgement here. But I think it’s obvious both from the actions of devs not rewriting their software, and python being willing to make breaking changes, and that everybody seems to be blaming each other (and have been for a decade), that the goal here wasn’t really to create long lasting tools.
commit b71c7dc9ddd6997be49ed6aaabf99a067e2c0388
Author: Victor Stinner <victor.stinner@gmail.com>
Date: Fri Oct 10 11:55:41 2014 +0200
Issue #22591: Drop support of MS-DOS
Drop support of MS-DOS, especially of the DJGPP compiler (MS-DOS port of GCC).
Today is a sad day. Good bye MS-DOS, good bye my friend :'-(But there are still large projects like motioneye where Python 3 port is WIP or not planned all.
Python REPL was my go to shell calculator and it is quite elaborate to put the parentheses around what I would like to print, for me that was the main drawback of python3.
But I don't write production code in python, it is used only for small shell scripts, or like calculator.
Printing to anything but stdout, printing with a separator, using the end keyword, etc, all became easier.
This way you don't even have to type "print" ;)
>>> 5+5
10Now Python 4.0 is coming out, and soon Python 3.X will lose support.
I think it's not even possible to compile it without a Windows XP VM but the output still runs on Windows 10.
Microsoft honestly should have provided a compatible version of Visual Basic that would run in the .NET environment instead of VB.NET. That would have allowed much more software to move over than what actually happened.
It is so ironic that in Python the most "unpythonic" thing is the version management of the language and libraries etc itself.
Something like python poetry, venv etc are great steps into the right direction, but there should be an official solution that "just works" and is predictable.
When I give python courses I usually skip dependency managment and explain it in the very end because it would probably (rightfully so) break the will of everybody involved. It shouldn't be like that
All the problems come before that point, with multiple installations making it so you don't know which installation you're using, which version, etc.
Also, many devs who work in Python in fact hate it. Python is so popular, it's foisted onto people.
In some jobs, you have to interact with Python from time to time, even if you don't develop in it.
print "hello" > print("hello")
let python be python3, and python2 for python2 instead
btw there is also a python-is-python3 package.
I would advice you to not use either of them, use virtualenv, pyenv or poetry instead.
I have had to rewrite several projects, just so they could continue working.
But I understand the need fot it. So iam not bitter or anything
> How long will PyPy support Python2?
> Since RPython is built on top of Python2 and that is extremely unlikely to change, the Python2 version of PyPy will be around “forever”, i.e. as long as PyPy itself is around.
https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-s...
One would hope that language designers would make the languages monotonically increasing to preserve compatibility and otherwise fork not projects when they loose confidence in previous features.
Edit: I should probably clarify. I have nothing against Python, quite the contrary, I've just always found that version 2 vs 3 thing odd.