From Fedora Project Wiki
m (Documentation)
 
(17 intermediate revisions by 4 users not shown)
Line 19: Line 19:


<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
 
= Retire Python 2 =
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
= Retire Python 2 (and let it all burn) =


== Summary ==
== Summary ==
Line 27: Line 25:
Note that motivation for the change should be in the Motivation section below, and this part should answer the question "What?" rather than "Why?". -->
Note that motivation for the change should be in the Motivation section below, and this part should answer the question "What?" rather than "Why?". -->


The {{package|python2}} package and all it's subpackages will be removed from Fedora 32.
The {{package|python2}} package and all its subpackages will be removed from Fedora 32.
A legacy {{package|python27}}  package for developers and users will be provided.
A legacy {{package|python27}}  package for developers and users will be provided.
All packages in Fedora that need Python 2 to run will be removed from Fedora 32 regardless of their dependencies.
All packages in Fedora that need Python 2 to run will be removed from Fedora 32 regardless of their dependencies.
Line 61: Line 59:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1708725 #1708725]
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/338 #338]


== Detailed Description ==
== Detailed Description ==
Line 67: Line 66:
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->


Python 2 is unsupported upstream since 2020-01-01. After two phased approaches to limit the dependent packages on Python 2:
Python 2 is unsupported upstream since 2020-01-01. Packages dependent on Python 2 are being removed from Fedora for several releases already:


* [[Changes/Mass_Python_2_Package_Removal|Fedora 30 Mass Python 2 Package Removal]]
* [[Changes/Mass_Python_2_Package_Removal|Fedora 30 Mass Python 2 Package Removal]]
* [[Changes/F31_Mass_Python_2_Package_Removal|Fedora 31 Mass Python 2 Package Removal]]
* [[Changes/F31_Mass_Python_2_Package_Removal|Fedora 31 Mass Python 2 Package Removal]]


The Python maintenance team has decided to pull the plug. The {{package|python2}} package and all it' subpackages will be retired (read: removed) from Fedora 32 (Rawhide) as soon as Fedora 31 is branched.
Now, the Python maintainers have decided to pull the plug. The {{package|python2}} package and all its subpackages will be retired (read: removed) from Fedora 32 (Rawhide) as soon as Fedora 31 is branched (in fact we'll wait until after the [[Changes/Python3.8]] side tag is merged).


All packages depending on any python2 package will be removed. The removal starts 2 week before the planned Fedora 32 Mass Rebuild.
All packages depending on any python2 package will be removed. '''The removal starts mid November 2019.'''
Broken dependencies will not stop the removals.
Broken dependencies will not stop the removals.
Packages that Fail to Build From Source and prevent to remove Python 2 subpackages may end up with broken dependencies,
Packages that Fail to Build From Source and prevent to remove Python 2 subpackages may end up with broken dependencies,
Line 81: Line 80:
The rules also apply to modules built for Fedora 32+.
The rules also apply to modules built for Fedora 32+.


The packages removal will be executed in an automated fashion.
The package removal will be executed in an automated fashion.


Removed packages that would otherwise block the upgrades to Fedora 32 will be obsoleted from {{package|fedora-obsolete-packages}}.
Removed packages that would block the upgrades to Fedora 32 will be obsoleted from {{package|fedora-obsolete-packages}}.


=== The python27 package ===
=== The python27 package ===


Similarly to existing {{package|python36}}, {{package|python37}} etc. packages, a {{package|python27}} package will be created.
Similarly to existing {{package|python36}}, {{package|python37}} etc. packages, a {{package|python27}} package will be created.
This package is indented for Python developers who still need to support the legacy version of Python.
This package is intended for Python developers who still need to support the legacy version of Python.
This package is indented for users, who still need to use some software depending on the legacy version of Python.
This package is intended for users, who still need to use some software depending on the legacy version of Python.
This package is not intended for other Fedora packages to be dependent upon.
This package is not intended for other Fedora packages to be depended upon.


The {{package|python27}} package has several drawbacks compared to the original {{package|python2}} package:
The {{package|python27}} package has several drawbacks compared to the original {{package|python2}} package:
Line 97: Line 96:
* there is no debug build (previously available as {{package|python2-debug}})
* there is no debug build (previously available as {{package|python2-debug}})
* there is no <code>/usr/bin/python</code> (note: there might be already the case before this change)
* there is no <code>/usr/bin/python</code> (note: there might be already the case before this change)
* any special backwards compatible provides are removed (this package is not intended to be dependent upon)
* any special backwards compatible Provides are removed (this package is not intended to be depended upon)


=== FESCo exceptions ===
=== FESCo exceptions ===
Line 103: Line 102:
We realize that there are some packages whose removal could seriously hurt Fedora. FESCo can grant exceptions for packages to use the {{package|python27}} as a runtime or build dependency.
We realize that there are some packages whose removal could seriously hurt Fedora. FESCo can grant exceptions for packages to use the {{package|python27}} as a runtime or build dependency.


The package maintainer is responsible to check the entire dependency chain and they need to request exception for the entire list of packages. For example, when seeking exception for the {{package|chromium}} package, it should contain {{package|python-psutil}} and other dependent packages. Yes, this is tedious. Maintaining a Python 2 dependent package is a burden. A Python 3 plan is a must for such exception.
The package maintainer is responsible to check the entire dependency chain and they need to request exceptions for the entire list of packages. For example, when seeking exception for the {{package|chromium}} package, the request should contain {{package|python-psutil}} and other dependent packages. (Yes, this is tedious. Maintaining a Python 2 dependent package is a burden.)
 
The exception request must include a plan for migrating to Python 3.


Any non-essential dependency must be dropped. That includes optional dependencies, test dependencies, optional subpackages etc.
Any non-essential dependency must be dropped. That includes optional dependencies, test dependencies, optional subpackages etc.


Package that fail to get an exception when the removal starts (see above) will be removed. Their importance for Fedora Release Engineering, Fedora Infrastructure or any other body will not be automagically respected, every package that needs Python 2 needs an exception.
Package that fail to get an exception when the removal starts (see above) will be removed. Their importance for Fedora Release Engineering, Fedora Infrastructure or any other body will not be automagically respected; every package that needs Python 2 needs an exception.


The change owners will send regular reminders to the package owners.
The change owners will send regular reminders to the package owners.
Line 145: Line 146:
* it leaves Python developers an option to use it in case they still need to support it
* it leaves Python developers an option to use it in case they still need to support it
* it leaves Fedora users an option to use it in case they still need it to run their (3rd party) software
* it leaves Fedora users an option to use it in case they still need it to run their (3rd party) software
* it leaves Fedora packagers an option to keep using it, arguably complicated but possible
* it leaves Fedora packagers an option to keep using it (complicated, but possible)


While:
While:
Line 151: Line 152:
* it removes Python 2 software from Fedora that was only preserved so far by inaction
* it removes Python 2 software from Fedora that was only preserved so far by inaction


Using Python 2 is dangerous. While the Fedora Python maintainers will try to fix as much security bugs as possible, without the upstream involvement, this will not always be possible. This change moves Python 2 from second class citizen to third class citizen. Python 2 was deprecated since Fedora 30.
Using Python 2 is dangerous. While the Fedora Python maintainers will try to fix as many security bugs as possible, without the upstream involvement this will be hard.
 
Python 2 is deprecated since Fedora 30. This change moves Python 2 from second class citizen to third class citizen.


== Scope ==
== Scope ==
* Proposal owners:
* Proposal owners:
** retire {{package|python2}}
** retire {{package|python2}} (done)
** introduce {{package|python27}}
** introduce {{package|python27}} (done)
** remove all {{package|python2}} dependent packages without exceptions
** remove all {{package|python2}} dependent packages that do not have FESCo exceptions (mid November 2019)
** obsolete removed packages that break the upgrade path via {{package|fedora-obsolete-packages}}
** obsolete removed packages that break the upgrade path via {{package|fedora-obsolete-packages}}
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
Line 167: Line 170:
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
* Release engineering: [https://pagure.io/releng/issue/8306 #8306] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.  
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.  
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
Line 173: Line 176:
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->


* Policies and guidelines: Python 2 packaging is against the guidelines since Fedora 30. Python 2 packaging guidelines will be removed from [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/ Python Appendix].<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines: Python 2 packaging is against the guidelines since Fedora 30. Python 2 packaging guidelines will be removed from [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/ Python Appendix] (unless the FPC wants to keep them around until F31 EOL).<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->


Line 184: Line 187:
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->


Majority of removed packages will be obsoleted and removed on upgrade.
The majority of removed packages will be obsoleted and removed on upgrade.


Users needing Python 2 libraries will need to use pip and virtualenv.
Users needing Python 2 libraries will not find these packaged as RPMs. They may install upstream versions using pip and virtualenv.


== How To Test ==
== How To Test ==
Line 231: Line 234:
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism:  
* Contingency mechanism:  
** In case of great madness, FESCo can issue a general exception for packages that would otherwise prevent Fedora 32 from being composed.
** In case of serious issues, FESCo can issue a general exception for packages that would otherwise prevent Fedora 32 from being composed.
** If someone steps up to maintain Python 2 (including the full ecosystem of packages now in Fedora), they can decide to discontinue removing packages, revert the Change, or come up with another plan. (Note that in this case, current maintainers will most likely orphan many fundamental python2 packages.)
** If someone steps up to maintain Python 2 (including the full ecosystem of packages now in Fedora), they can decide to discontinue removing packages, revert this Change, or come up with another plan. (Note that in this case, current maintainers will most likely orphan many fundamental python2 packages.)
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
Line 245: Line 248:
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
This page should serve as the documentation.
This page should serve as the documentation.
=== See Also ===
* [https://www.python.org/doc/sunset-python-2/ Sunsetting Python 2]


== Release Notes ==
== Release Notes ==
Line 253: Line 260:
-->
-->


[[Category:ChangePageIncomplete]]
TBD.
 
[[Category:ChangeAcceptedF32]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Latest revision as of 16:45, 12 September 2019


Retire Python 2

Summary

The python2 package and all its subpackages will be removed from Fedora 32. A legacy python27 package for developers and users will be provided. All packages in Fedora that need Python 2 to run will be removed from Fedora 32 regardless of their dependencies. All packages in Fedora that need Python 2 to build will be removed from Fedora 32 regardless of their dependencies. Exceptions can be granted by FESCo.

Owner

  • Name: Miro Hrončok
  • Email: <mhroncok@redhat.com> <python-devel@lists.fedoraproject.org>
  • Release notes owner:

Current status

Detailed Description

Python 2 is unsupported upstream since 2020-01-01. Packages dependent on Python 2 are being removed from Fedora for several releases already:

Now, the Python maintainers have decided to pull the plug. The python2 package and all its subpackages will be retired (read: removed) from Fedora 32 (Rawhide) as soon as Fedora 31 is branched (in fact we'll wait until after the Changes/Python3.8 side tag is merged).

All packages depending on any python2 package will be removed. The removal starts mid November 2019. Broken dependencies will not stop the removals. Packages that Fail to Build From Source and prevent to remove Python 2 subpackages may end up with broken dependencies, in cases where it is not desired, those packages will be retired instead.

The rules also apply to modules built for Fedora 32+.

The package removal will be executed in an automated fashion.

Removed packages that would block the upgrades to Fedora 32 will be obsoleted from fedora-obsolete-packages.

The python27 package

Similarly to existing python36, python37 etc. packages, a python27 package will be created. This package is intended for Python developers who still need to support the legacy version of Python. This package is intended for users, who still need to use some software depending on the legacy version of Python. This package is not intended for other Fedora packages to be depended upon.

The python27 package has several drawbacks compared to the original python2 package:

  • it is "flat" - there are no subpackages, everything lives in one package
  • there is no debug build (previously available as python2-debug)
  • there is no /usr/bin/python (note: there might be already the case before this change)
  • any special backwards compatible Provides are removed (this package is not intended to be depended upon)

FESCo exceptions

We realize that there are some packages whose removal could seriously hurt Fedora. FESCo can grant exceptions for packages to use the python27 as a runtime or build dependency.

The package maintainer is responsible to check the entire dependency chain and they need to request exceptions for the entire list of packages. For example, when seeking exception for the chromium package, the request should contain python-psutil and other dependent packages. (Yes, this is tedious. Maintaining a Python 2 dependent package is a burden.)

The exception request must include a plan for migrating to Python 3.

Any non-essential dependency must be dropped. That includes optional dependencies, test dependencies, optional subpackages etc.

Package that fail to get an exception when the removal starts (see above) will be removed. Their importance for Fedora Release Engineering, Fedora Infrastructure or any other body will not be automagically respected; every package that needs Python 2 needs an exception.

The change owners will send regular reminders to the package owners.

Benefit to Fedora

Python 2 is past upstream End of Life since 2020-01-01. This changes is generally crafted in a way that:

  • it leaves Python developers an option to use it in case they still need to support it
  • it leaves Fedora users an option to use it in case they still need it to run their (3rd party) software
  • it leaves Fedora packagers an option to keep using it (complicated, but possible)

While:

  • it removes Python 2 software from Fedora that was only preserved so far by inaction

Using Python 2 is dangerous. While the Fedora Python maintainers will try to fix as many security bugs as possible, without the upstream involvement this will be hard.

Python 2 is deprecated since Fedora 30. This change moves Python 2 from second class citizen to third class citizen.

Scope

  • Proposal owners:
  • Other developers:
    • remove their python2 dependent packages without exceptions
    • get exceptions if needed
    • fix broken dependencies
  • Policies and guidelines: Python 2 packaging is against the guidelines since Fedora 30. Python 2 packaging guidelines will be removed from Python Appendix (unless the FPC wants to keep them around until F31 EOL).
  • Trademark approval: not needed for this Change

Upgrade/compatibility impact

The majority of removed packages will be obsoleted and removed on upgrade.

Users needing Python 2 libraries will not find these packaged as RPMs. They may install upstream versions using pip and virtualenv.

How To Test

Try to update Fedora 30 or 31 to 32. No python2 packages should block the upgrade. Try to run Python 2 software via the python27 package.

User Experience

There will be close to zero Python 2 RPMs in Fedora repos. Users are encouraged to switch to Python 3 and/or use Python 2 virtual environments and pip for development.

Dependencies

Ideally, all programs that use python2 would be switched to use python3. Although we don't expect everything to be switched over, as much as possible should be, so that the ripped remaining python2 set is small as possible.

Contingency Plan

  • Contingency mechanism:
    • In case of serious issues, FESCo can issue a general exception for packages that would otherwise prevent Fedora 32 from being composed.
    • If someone steps up to maintain Python 2 (including the full ecosystem of packages now in Fedora), they can decide to discontinue removing packages, revert this Change, or come up with another plan. (Note that in this case, current maintainers will most likely orphan many fundamental python2 packages.)
  • Contingency deadline: Fedora 32 Beta
  • Blocks release? in theory it should not, in practice, it may break the release and hence it will block it until fixed
  • Blocks product? all of them?

Documentation

This page should serve as the documentation.

See Also

Release Notes

TBD.

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/RetirePython2&oldid=552472"