Latest revision as of 10:03, 8 October 2015

Python 3 as the Default Implementation

Summary

Up until now, Fedora has used Python 2 as the default Python implementation. This change proposes switching to Python 3. The details of the term "switching" are explained thoroughly in the Scope section.

Owner

Name: Slavek Kabrda
Email: bkabrda@redhat.com
Name: Matej Stuchlik
Email: mstuchli@redhat.com
Name: Miro Hroncok
Email: mhroncok@redhat.com
Name: Thomas Spura
Email: tomspur [at] fedoraproject.org
Name: Robert Kuska
Email: rkuska [at] redhat.com
Release notes owner:

Current status

Targeted release: Fedora 23
Last updated: 22-03-2015
Tracker bug: #1076441 (we already created a tracker bug: https://bugzilla.redhat.com/show_bug.cgi?id=1014209)

Detailed Description

Python 3 is the next generation of Python programming language. It is currently mature and stable, since it has been under active development for five years - version 3.0 was released in December 2008, current latest stable version is 3.4.3 released in February 2015. The main reason to switch to Python 3 as the default implementation is that Python 2 is in maintenance mode, thus only bugfixes and security fixes are accepted upstream. Further reasons are mentioned in the Benefit to Fedora section. For this Change to be carried out successfully, it is necessary that the key packages in the Fedora software stack be ported to Python 3. These are parts of the minimal buildroot, the default package manager, programs present on the LiveCD etc. More information on the packages involved can be found in Dependencies. While porting of some packages is rather trivial, other packages need significant amount of work to get rid of the Python 2 dependence.

Benefit to Fedora

Python 2.7 (latest Python 2 release, which we also have in Fedora) is currently in maintenance mode only, which means upstream only accepts bugfixes and security fixes, but no new features are being implemented. Python 2.7 is the last minor release of Python 2, with upstream support until 2020. Python 3, on the other hand, is actively developed and new features are being added every release. Moreover, there is currently no end of support date for Python 3.

Fedora already has Python 3 stack that is parallel to Python 2 stack. There are several benefits of switching the "primary" Python stack:

Getting upstream support for default system version will not be limited by time.
Our system tools will be able to switch to Python 3, drop the burden of Python 2 support and use new features of Python 3.
As a distribution that stays close to upstream, Fedora should help Python community go forward by contributing patches and working closely with upstreams to get this accomplished. Thus this Change is meant to benefit not only Fedora, but also broader Python community.
Switching to Python 3 as a default will once again push Fedora to stay as close to upstream as possible, highlighting the "Features" and "First" (although, to be honest, Arch Linux was first in this...)

Scope

The main goal is switching to Python 3 as a default, in which state:

DNF is the default package manager instead of Yum, which only works with Python 2
Python 3 is the only Python implementation in the minimal buildroot (already done since F22)
Python 3 is the only Python implementation on the Workstation LiveDVD
Python 3 is the only Python implementation in minimal cloud image
Python 3 is the only Python implementation in Atomic host
It'd also be nice to have Python 3 as the only implementation on the Server LiveDVD, but it's not likely

(see Dependencies for the list of packages that need to be ported)

Changes in packaging:

Change definition of default python interpreter to python3.
Change unversioned python macros to python3 (Possibly do that change in upstream rpm)
All applications that use only a single python version MUST use python3 (unless they have a good reason not to do so).

Fate of /usr/bin/python:

/usr/bin/python will still point to Python2 version of interpreter as suggested in PEP0394.
There will be no /usr/bin/python on LiveDVD as python3 will became the only python interpreter shipped by default.
Python (python2 version of the interpreter) package will be still available (via dnf install) and will provide /usr/bin/python.

There are basically two types of packages that need to undergo the conversion:

"libraries" - Python extension modules and libraries that provide Python bindings - assuming that there is upstream support, these can receive python3- subpackage any time without any damage to Fedora; we can then just utilize this subpackage when switching to Python 3 (instead of using current python- subpackage).
"applications" - Packages that build with some sort of "embedded Python support", like gdb, or Rhythmbox with its plugins. In these cases, it makes no sense to do a python3- subpackage, since the whole package would need to be duplicated (e.g. python3-gdb). These packages should be built with Python 3 in Fedora 23 rawhide as soon as possible.

Work in Fedora 21 Timeframe

Proposal owners:
- Discussing changes in Python packaging guidelines with Fedora community and FPC
- Helping upstreams with porting to Python 3
- Introducing python3- packages where appropriate, testing packages that only build with Python once (e.g. gdb, Rhythmbox)

Other developers:
- Hopefully the same as proposal owners.

Release engineering:
- Nothing.

Policies and guidelines:
- None for F21

Work in Fedora 22 Timeframe

Proposal owners:
- Discussing changes in Python packaging guidelines with Fedora community and FPC
- Helping upstreams with porting to Python 3
- Introducing python3- packages where appropriate, testing packages that only build with Python once (e.g. gdb, Rhythmbox)

Other developers:
- Hopefully the same as proposal owners.

Release engineering:
- Nothing.

Policies and guidelines:
- None for F22

Work in Fedora 23 Timeframe

Proposal owners:
- Continue the work from F21 and F22 timeframe
- Modify comps accordingly
- Apply the changes to Python packaging guidelines

Other developers:
- Introduce python3- subpackages where appropriate, build against Python 3 if the package supports it

Release engineering:
- Nothing

Policies and guidelines:
- TODO: changes in packaging

Upgrade/compatibility impact

The Python 2 stack will stay in Fedora, it will just not be the default one. Depending on the modifications done to Python packaging guidelines, this probably means that Python 2 packages will stay the way they are and default tools will drag in python3- dependencies. Upstream recommends that /usr/bin/python point to Python 2 runtime for the time being, so if we go with that, there shouldn't be any serious compatibility impact:

Users will still be able to install Python 2 packages
Both Python 2 and 3 stacks will still live in parallel

Libraries that are built with Python only once (like gdb) may force users to rewrite their custom scripts and plugins (see e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1014549#c3).

How To Test

No special hardware is needed.
Install Fedora 23 in mock/virtual machine and test if everything still works, most importantly system tools like DNF or Firewalld.
Test installing the system with Anaconda running on Python 3.

User Experience

Users shouldn't notice any changes, except that packages in minimal buildroot and on LiveCD will be python3-, not python-. Anaconda deps will be python3- as well. /usr/bin/python will still point to Python 2 and depending on result of discussions with FPC, "yum install python-foo" will still install Python 2 version of the package.

Dependencies

See https://fedoraproject.org/wiki/User:Churchyard/python3 (our tracking page with notes) or https://bugzilla.redhat.com/show_bug.cgi?id=1014209 (tracking bug).

Contingency Plan

Contingency mechanism: None needed. Packages that will be ready will be built with Python 3, the rest will be ported in next release.
Contingency deadline: Software string freeze
Blocks release? No

Documentation

https://docs.python.org/3.0/whatsnew/3.0.html

https://docs.python.org/3.1/whatsnew/3.1.html

https://docs.python.org/3.2/whatsnew/3.2.html

https://docs.python.org/3.3/whatsnew/3.3.html

https://docs.python.org/3.4/whatsnew/3.4.html

https://docs.python.org/3.5/whatsnew/3.5.html

http://en.wikipedia.org/wiki/History_of_Python#Version_3.0

http://docs.python.org/dev/howto/pyporting.html

http://docs.python.org/3/howto/cporting.html

https://wiki.gnome.org/PyGObject/IntrospectionPorting

https://lwn.net/Articles/426906/

https://lwn.net/Articles/650904/

Release Notes

@@ Line 13: / Line 13: @@
 * Name: [[User:Mstuchli|Matej Stuchlik]]
 * Email: mstuchli@redhat.com
+* Name: [[User:Churchyard|Miro Hroncok]]
+* Email: mhroncok@redhat.com
+* Name: [[User:Tomspur|Thomas Spura]]
+* Email: tomspur [at] fedoraproject.org
+* Name: [[User:rkuska|Robert Kuska]]
+* Email: rkuska [at] redhat.com
 * Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 <!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
@@ Line 19: / Line 25: @@
 == Current status ==
-* Targeted release: [[Releases/22 | Fedora 22 ]]
+* Targeted release: [[Releases/23 | Fedora 23 ]]
-* Last updated: 10-03-2013
+* Last updated: 22-03-2015
 <!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page
 Bugzilla states meaning as usual:
@@ Line 29: / Line 35: @@
 CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 -->
-* Tracker bug: <will be assigned by the Wrangler> (we already created a tracker bug: https://bugzilla.redhat.com/show_bug.cgi?id=1014209)
+* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1076441 #1076441] (we already created a tracker bug: https://bugzilla.redhat.com/show_bug.cgi?id=1014209)
 == Detailed Description ==
-Python 3 is the next generation of Python programming language. It is currently mature and stable, since it has been under active development for five years - version 3.0 was released in December 2008, current latest stable version is 3.3.2 released in May 2013. The main reason to switch to Python 3 as the default implementation is that Python 2 is in maintenance mode, thus only bugfixes and security fixes are accepted upstream. Further reasons are mentioned in the [[#Benefit to Fedora|Benefit to Fedora]] section.
+Python 3 is the next generation of Python programming language. It is currently mature and stable, since it has been under active development for five years - version 3.0 was released in December 2008, current latest stable version is 3.4.3 released in February 2015. The main reason to switch to Python 3 as the default implementation is that Python 2 is in maintenance mode, thus only bugfixes and security fixes are accepted upstream. Further reasons are mentioned in the [[#Benefit to Fedora|Benefit to Fedora]] section.
 For this Change to be carried out successfully, it is necessary that the key packages in the Fedora software stack be ported to Python 3. These are parts of the minimal buildroot, the default package manager, programs present on the LiveCD etc. More information on the packages involved can be found in [[#Dependencies|Dependencies]]. While porting of some packages is rather trivial, other packages need significant amount of work to get rid of the Python 2 dependence.
 == Benefit to Fedora ==
 <!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->
-Python 2.7 (latest Python 2 release, which we also have in Fedora) is currently in maintenance mode only, which means upstream only accepts bugfixes and security fixes, but no new features are being implemented. According to upstream, Python 2.7 is the last release of Python 2 (http://www.python.org/dev/peps/pep-0404/). Support for Python 2 is guaranteed by upstream until May 2015 (http://www.python.org/dev/peps/pep-0373/#maintenance-releases), then it may continue for some time, or it may not. Python 3, on the other hand, is actively developed and new features are being added every release. Moreover, there is currently no end of support date for Python 3.
+Python 2.7 (latest Python 2 release, which we also have in Fedora) is currently in maintenance mode only, which means upstream only accepts bugfixes and security fixes, but no new features are being implemented. Python 2.7 is [https://www.python.org/dev/peps/pep-0404/ the last minor release of Python 2], with upstream support [https://www.python.org/dev/peps/pep-0373/#maintenance-releases until 2020]. Python 3, on the other hand, is actively developed and new features are being added every release. Moreover, there is currently no end of support date for Python 3.
-Fedora already has Python 3 stack that is parallel to Python 2 stack. The are several benefits of switching the "primary" Python stack:
+Fedora already has Python 3 stack that is parallel to Python 2 stack. There are several benefits of switching the "primary" Python stack:
 * Getting upstream support for default system version will not be limited by time.
 * Our system tools will be able to switch to Python 3, drop the burden of Python 2 support and use new features of Python 3.
@@ Line 49: / Line 55: @@
 The main goal is switching to Python 3 as a default, in which state:
 * DNF is the default package manager instead of Yum, which only works with Python 2
-* Python 3 is the only Python implementation in the minimal buildroot
+* Python 3 is the only Python implementation in the minimal buildroot (already done since F22)
-* Python 3 is the only Python implementation on the LiveCD
+* Python 3 is the only Python implementation on the Workstation LiveDVD
-* Anaconda and all of its dependencies run on Python 3
+* Python 3 is the only Python implementation in minimal cloud image
-* cloud-init and all of its dependencies run on Python 3
+* Python 3 is the only Python implementation in Atomic host
+* It'd also be nice to have Python 3 as the only implementation on the Server LiveDVD, but it's not likely
 (see [[#Dependencies|Dependencies]] for the list of packages that need to be ported)
-This will also require revisiting Python guidelines (broader discussion with community and FPC approval - TBD). The result of the discussion will reflect in this Change in further instructions for Fedora packagers.
+Changes in packaging:
+* Change definition of default python interpreter to python3.
+* Change unversioned python macros to python3 (Possibly do that change in [http://rpm.org/ticket/884 upstream rpm])
+* All applications that use only a single python version MUST use python3 (unless they have a good reason not to do so).
+Fate of /usr/bin/python:
+* /usr/bin/python will still point to Python2 version of interpreter as suggested in [https://www.python.org/dev/peps/pep-0394/ PEP0394].
+* There will be no /usr/bin/python on LiveDVD as python3 will became the only python interpreter shipped by default.
+* Python (python2 version of the interpreter) package will be still available (via dnf install) and will provide /usr/bin/python.
 There are basically two types of packages that need to undergo the conversion:
-* Python extension modules and libraries that provide Python bindings - assuming that there is upstream support, these can recieve python3- subpackage anytime without any damage to Fedora; we can then just utilize this subpackage when switching to Python 3 (instead of using current python- subpackage).
+* "libraries" - Python extension modules and libraries that provide Python bindings - assuming that there is upstream support, these can receive python3- subpackage any time without any damage to Fedora; we can then just utilize this subpackage when switching to Python 3 (instead of using current python- subpackage).
-* Packages that build with some sort of "embedded Python support", like gdb, or Rhythmbox with its plugins. In these cases, it makes no sense to do a python3- subpackage, since the whole package would need to be duplicated (e.g. python3-gdb). These packages should be tested with Python 3 locally without any modifications to how they're currently built in Fedora. When we get a Koji side tag, these packages will switch and build against Python 3 in the side tag.
+* "applications" - Packages that build with some sort of "embedded Python support", like gdb, or Rhythmbox with its plugins. In these cases, it makes no sense to do a python3- subpackage, since the whole package would need to be duplicated (e.g. python3-gdb). These packages should be built with Python 3 in Fedora 23 rawhide as soon as possible.
 === Work in Fedora 21 Timeframe ===
@@ Line 71: / Line 89: @@
 * Release engineering:
-** Nothing in F21 timeframe
+** Nothing.
 * Policies and guidelines:
-** As mentioned above, this will require a discussion with community and FPC and preparation of changes to Python packaging guidelines. The changes related to the actual switch should however be merged in F22 timeframe and only if the switch is successful.
+** None for F21
 === Work in Fedora 22 Timeframe ===
 * Proposal owners:
-** Continue the work from F21 timeframe
+** Discussing changes in Python packaging guidelines with Fedora community and FPC
-** Request Koji side tag and encourage packagers to rebuilt their packages with Python 3 there
+** Helping upstreams with porting to Python 3
-** If the switch to Python 3 is achieved in the side tag:
+** Introducing python3- packages where appropriate, testing packages that only build with Python once (e.g. gdb, Rhythmbox)
-*** Modify comps accordingly
-*** Apply the changes to Python packaging guidelines
+* Other developers:
-*** Ask relengs to merge the side tag into F22
+** Hopefully the same as proposal owners.
-** Else:
-*** Postpone the Change for another release
+* Release engineering:
-*** Do not merge side tag into F22
+** Nothing.
-*** Do not apply changes to Python packaging guidelines
+* Policies and guidelines:
+** None for F22
+=== Work in Fedora 23 Timeframe ===
+* Proposal owners:
+** Continue the work from F21 and F22 timeframe
+** Modify comps accordingly
+** Apply the changes to Python packaging guidelines
 * Other developers:
-** Introduce python3- subpackages where appropriate, build against Python 3 in the side tag where appropriate
+** Introduce python3- subpackages where appropriate, build against Python 3 if the package supports it
 * Release engineering:
-** Create the Koji side tag
+** Nothing
-** Merge the side tag back in F22 if the switch is achieved
 * Policies and guidelines:
-** Apply the changes to Python guidelines if the switch is achieved
+** TODO: changes in packaging
 == Upgrade/compatibility impact ==
@@ Line 125: / Line 150: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 * No special hardware is needed.
-* Using mock for testing is strongly recommended. During rebuilding in the Koji side tag, things may get unstable and you'd be risking breaking your system.
+* Install Fedora 23 in mock/virtual machine and test if everything still works, most importantly system tools like DNF or Firewalld.
-* Install the packages built against Python 3 from the Koji side tag and test if they still work as expected.
+* Test installing the system with Anaconda running on Python 3.
-* When the switch is prepared in the side tag, just try installing the packages from there and removing Python 2 from your system. Again, look for any problems that may be related to the switch.
 == User Experience ==
@@ Line 142: / Line 166: @@
 == Contingency Plan ==
 <!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
-* Contingency mechanism: Don't merge the koji side tag into F22, postpone the feature to F23, keep the side tag with all the work that has been done.
+* Contingency mechanism: None needed. Packages that will be ready will be built with Python 3, the rest will be ported in next release.
 <!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
 * Contingency deadline: Software string freeze
@@ Line 152: / Line 176: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+https://docs.python.org/3.0/whatsnew/3.0.html
+https://docs.python.org/3.1/whatsnew/3.1.html
+https://docs.python.org/3.2/whatsnew/3.2.html
+https://docs.python.org/3.3/whatsnew/3.3.html
+https://docs.python.org/3.4/whatsnew/3.4.html
+https://docs.python.org/3.5/whatsnew/3.5.html
+http://en.wikipedia.org/wiki/History_of_Python#Version_3.0
 http://docs.python.org/dev/howto/pyporting.html
@@ Line 157: / Line 196: @@
 https://wiki.gnome.org/PyGObject/IntrospectionPorting
+https://lwn.net/Articles/426906/
+https://lwn.net/Articles/650904/
 == Release Notes ==
@@ Line 165: / Line 208: @@
 -->
-[[Category:ChangeAnnounced]]
+[[Category:ChangeAcceptedF23]]
 <!-- When your change proposal page is completed and ready for review and announcement -->
 <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
@@ Line 174: / Line 217: @@
 <!-- [[Category:SelfContainedChange]] -->
 [[Category:SystemWideChange]]
+[[Category:Python]]

Search

Changes/Python 3 as Default: Difference between revisions