RFC 77: Drop Python 2 support in favour of Python 3.6
Author: |
Idan Miara |
Contact: |
|
Started: |
2020-Nov-3 |
Last updated: |
2020-Nov-29 |
Status: |
Adopted, implemented in GDAL 3.3 |
Summary
This RFC drops Python 2 support and sets Python 3.6 as the new minimum supported Python version.
Motivation
Currently GDAL Python bindings support Python 2.7 and Python 3 (so only the common between the two). Python 2 is at End Of Life, and is no longer supported since the January 2020. https://www.python.org/doc/sunset-python-2/
"We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020."
While keeping Python 2.7 support might serve those who didn't upgrade their code to Python 3 in the 12 year transition period, This PR suggests that the time has come and the benefits of dropping Python 2 support outnumber the drawbacks. Virtually all supported OS and relevant programs already use Python 3. Moreover, most of the related projects that usually is used with GDAL already dropped Python 2 support (as can be seen below). It makes sense that people who didn't upgrade their code in 12 years are still using a much older version of GDAL anyway...
The drawbacks of keeping Python 2 support puts unnecessary maintenance burden on the GDAL maintainers, As maintainers need to make sure their new code is backwards compatible with Python 2. Furthermore, many important features that added in Python 3 cannot be used in GDAL to maintain backwards compatibility with Python 2.
Python 3 version status:
Python 3.5 (Released: Sep 2015) is End Of Life. Many related projects already dropped support of it.
Python 3.6 (Released: Dec 2016) will be End Of Life on Dec 2021. Python 3.6 brings many important features.
Python 3.7 (Released: Jun 2018) will be End Of Life on Jun 2023. Python 3.7 is already widely adopted and supported by all the related projects listed above.
Python 3.8 (Released: Oct 2019) will be End Of Life on Oct 2024. Python 3.8 is too new to be set as the minimum supported version.
Python 3 OS Support:
Linux:
Distribution |
Release date |
End Of Life |
Python Version |
Ubuntu 16.04 Xenial LTS |
April 2016 |
April 2021 |
Python 3.5 |
Ubuntu 18.04 Bionic LTS |
April 2018 |
April 2023 |
Python 3.6 (3.7 in universe) |
Ubuntu 20.04 Xenial LTS |
April 2020 |
April 2021 |
Python 3.8 |
Debian 9.0 Stretch LTS |
June 2017 |
July 2022 |
Python 3.5 |
Debian 10.0 Buster LTS |
July 2019 |
~2022 |
Python 3.7 |
Centos/RHEL 8 |
September 2019 |
? |
Python 3.6 (?) |
Amazon Linux |
December 2021 |
Python 3.6 |
https://wiki.python.org/moin/Python3LinuxDistroPortingStatus
Windows:
Conda: Python 3.6-3.9
OSGeo4W: Python 3.7
gisinternals: Python 2.7, 3.4-3.7
MacOS:
It appears the even though Python 2.7 is preinstalled on Mac OS X, Python 3.5-3.9 can be installed alongside Python 2 on Mac OS X 10.8 (Released: July 2012) and newer.
Which version should be the new minimum version ?
This RFC suggest the new minimum supported Python version should be 3.6 for the following reasons:
Python < 3.6 is End of Life.
- Long list of great new features that were introduced in Python 3.6, Several of which are immediately useful to simplify code or improve testing:
f-strings
builtin pathlib
underscores in numeric literals
type annotations
malloc debugging
Python 3.6 is supported out of the box in virtually every relevant OS.
Python 3.6 is probably the safest choice for now in respect to other related projects.
Python 3.7 (and newer) isn't available seamlessly in some popular LTS Linux distributions.
We want to make the transition as smooth and easy as possible. Setting the minimum to Python 3.7 might make the transition harder for the CI because of the above reason.
Dropping Python 3.6 in favour of Python 3.7 or newer in future versions shouldn't be as hard as this drop (see next section for a suggested approach).
GDAL Release cycle and regular Python version dropping
When releasing GDAL 3.1.0, Even Rouault suggested GDAL would use fixed release cycles of 6 months between major versions:
Projecting from that suggestion, GDAL 3.3.0 should be released around April-May 2021.
We could potentially synchronize with NEP 29 - Recommend Python and Numpy version support as a community policy standard. Which suggests when to drop each Python version.
https://numpy.org/neps/nep-0029-deprecation_policy.html
NEP 29 suggests to drop support for Python 3.6 support on Jun 23, 2020 (in favour of Python 3.7).
We could potentially discuss similar/more conservative approaches and delay each drop by a few more months, or only drop Python versions that have reached End Of Life (As of today, Python < 3.6 have reached End Of Life). Further discussion on the matter of dropping other Python versions is a subject for another RFC.
Backward compatibility
Currently, GDAL Python code itself is compatible with Python 2 and Python 3. Once this PR is accepted, GDAL 3.3.0 would not be compatible with Python 2. Thus any "Python 2 only" code that uses GDAL would need to be upgraded to Python 3 and at the same time the respective Python interpreter would need to be upgraded to a supported Python version.
Will GDAL 3.2 be a LTS?
Currently - No. So far, nobody has stepped up to make a LTS, So there won't be one unless someone takes it up upon themselves or raise funds to make it happen. GDAL only provide bugfix releases of the current stable branch for 6 months.
CI Impacts:
Impacts on our CI should be analyzed. It seems that all our CI builds use Python 2.7 or 3.5, so all of them would need to be adjusted. In particular, builds that use older Linux distributions would need to be upgraded.
Impacts on GDAL core
There should be no impacts on GDAL core, As the Python bindings are generated by SWIG on top of the binary form of GDAL.
Limitations and scope
The scope of this RFC should be the GDAL Python code alone. There shouldn't be effect on any other language supported by GDAL.
SWIG binding changes
To begin with, the SWIG Python bindings already support Python 3.6. Dropping Python 2 support might allow us to use a newer SWIG version or to make some improvements to the bindings, but it doesn't have to be in the first step.
Security implications
Python 3.6 is the minimum Python version that is not End Of Life, thus still receiving security updates.
Performance impact
There might be some performance gain for this upgrade for some uses as there were many performance improvements between Python 2.7-3.6. The scope of the improvements could be limited because most of GDAL Python code is a thin wrapper around the C++ code.
Documentation
The GDAL Python documentation is generated automatically in should already support Python 3. If there are sections in the documentation that are Python 2 specific, they should be removed or refactored.
Testing
While upgrading the CI, Python 2 tests should be removed or upgraded. A simple test that fails on Python < 3.6 should be added. No any additional tests should be needed.
Previous discussions
This topic has been discussed in the past in :
Voting history
https://lists.osgeo.org/pipermail/gdal-dev/2020-November/053039.html
+1 from EvenR, HowardB, KurtS, JukkaR, DanielM
Credits
implemented by Even Rouault, Robert Coup and Idan Miara