Python 2’s end of life is fast approaching and the UK’s National Cyber Security Centre has issued a warning to developers still clinging on to Python 2 of risks they will face as a result of no more bug fixes or security updates.
Much to the dismay of Python 3 adherents, Python 2, was given a stay of execution by Guido van Rossum in 2014. While being adamant that there would be no Python 2.8 and that the time had come to move to Python 3.4, he announced at PyCon that instead of facing end of support in 2015 as originally planned, Python 2 was being granted an extra 5 years to 2020. This was in order to give time for numerous libraries relied on by existing projects to add Python 3 support.
In March 2018 it was agreed that Python 2.7, the only version still supported, would be completely dead on January 1st 2020, meaning no updates, not even source-only security patches, after that date. And if you think 2020 is still a long way off – you are wrong:
In view of this looming deadline, the National Cyber Security Centre (NCSC) has repeated the “time to move Python 3” message, saying:
So, if you’re still using 2.x, it’s time to port your code to Python 3. If you continue to use unsupported modules, you are risking the security of your organisation and data, as vulnerabilities will sooner or later appear which nobody is fixing.
Given that Python 3.0 was released in December 2008 and wasn’t backward compatible with the 2.x line of releases, this advice seems long overdue. But the Python community has been highly resistant to change. It took, four years, until January 2013, i.e. 4 years, for the number of monthly downloads of the latest Python 3 to exceed that of its Python 2 counterpart.
The main barrier to switching existing code from Python 2 to Python 3 was their dependencies on third party packages. At the time of Python 2’s EOL extension there were still a substantial number of widely used libraries that didn’t support Python 3.x. This situation was monitored by on a website initially called the Python 3 Wall of Shame as it displayed in red the names of PyPi packages that were not compatible Python using green for those that did. It was renamed the Python 3 Wall of Superpowers as more and more green entries replaced red ones and, having achieved over 95% compatibility, stopped the exercise in April 2018.
Even so some users still cling to Python 2.x. Figures from the 2018 Python Developers Survey conducted jointly by the Python Software Foundation and JetBrains.show that almost a fifth of Python developers engaged in DevOps are stuck there, with almost as many web developers in the same situation. The outlook is much better for Data Science, where only 10% have yet to upgrade.
These figures look reasonably reassuring, but a different picture is revealed by June 2019 stats of downloads of popular packages from the Python Package Index collated by NCSC:
The four of the packages listed at the top of the table had more downloads for Python 2 than Python 3 and even where Python 3 is more popular a substantial proportion are still for Python 2. To force this situation to improve many projects including NumPy, Requests, and TensorFlow have pledged to drop support for 2.x by 2020 and some already have. As NCSC points out,
This means that if you want to use the latest features of your favourite modules, you’ll need to be using Python 3. The longer you wait to update, the more the Python 3 versions of your dependencies will have changed, and the more difficult updating will become.
Another point raised by NCSC is that failure to move on is holding other developers back, stating:
If you maintain a library that other developers depend on, you may be preventing them from updating to 3. By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others.
It also has some recommendations to assist in the process of porting Python 2.x code to Python 3, mentioning:
Can I Use Python 3 – a program that checks your project dependencies to see if any are preventing you from using Python 3.
2to3 – a Python program, usually installed with the Python interpreter as a script, that attempts to convert 2.x source code into 3. Note that this isn’t perfect, you may still have to fix some code manually.
There are plenty of features in Python 3 to reward those who make the move and in doing so you will also have the opportunity to improve how you manage your software dependencies and minimize your security debt.
Dropbox migrated its codebase to Python 3 in 2018 and gave details of the experience in a blog post. The move was motivated by the fact that as Python 2 aged, the set of toolchains initially compatible for deploying it had largely become obsolete, leading to a growing maintenance burden:
- The use of older compilers/runtimes was limiting our ability to upgrade some important dependencies.
- For example, we use Qt on Windows and Linux: Recent versions of Qt require more modern compilers due to the inclusion of Chromium (via QtWebEngine).
- As we continued to integrate deeply with the operating system, our inability to rely on more recent versions of these toolchains increased the cost of adoption for newer APIs.
- For example, Python 2 still technically requires Visual Studio 2008. This version is no longer supported by Microsoft and is not compatible with the Windows 10 SDK.