Latest revision as of 08:06, 21 October 2020

Python 3.9

Summary

Update the Python stack in Fedora from Python 3.8 to Python 3.9.

Owner

Name: Miro Hrončok
Name: Charalampos Stratakis

Email: python-maint@redhat.com
Release notes owner:

Current status

Targeted release: Fedora 33
Last updated: 2020-10-21
Tracker bug: #1785415
Release notes tracker: #424

Detailed Description

We would like to upgrade Python to 3.9 in Fedora 33 thus we are proposing this plan early.

See the upstream notes at Features for 3.9 and What's new in 3.9.

Important dates and plan

2019-06-04: Python 3.9 development begins
2019-11-19: Python 3.9.0 alpha 1
- Package it as python3.9 for testing purposes
- Start the bootstrap procedure in Copr
- Do a mass rebuild against every future release in Copr
2019-12-18: Python 3.9.0 alpha 2
2020-01-25: Python 3.9.0 alpha 3
2020-02-11: Branch Fedora 32, Rawhide becomes future Fedora 33
- The earliest point when we can start rebuilding in Koji side-tag
2020-02-26: Python 3.9.0 alpha 4
2020-03-23: Python 3.9.0 alpha 5
2020-04-28: Python 3.9.0 alpha 6
2020-05-18: Python 3.9.0 beta 1
- No new features beyond this point
- The ideal point when we can start rebuilding in Koji
2020-05-29: Actual side tag-merge
2020-05-30: Expected side tag-merge (optimistic)
2020-06-09: Python 3.9.0 beta 2/3
2020-06-18: Expected side tag-merge (realistic)
2020-06-29: Python 3.9.0 beta 4
2020-07-18: Expected side tag-merge (pessimistic)
2020-07-20: Python 3.9.0 beta 5
2020-07-22: Fedora 33 Mass Rebuild
- The mass rebuild happens with fifth beta. We might need to rebuild Python packages later in exceptional case.
- If the Koji side-tag is not merged yet at this point, we defer the change to Fedora 34.
2020-08-10: Python 3.9.0 candidate 1
- This serves as "final" for our purposes.
2020-08-11: Branch Fedora 33, Rawhide becomes future Fedora 34
2020-08-11: Fedora 33 Change Checkpoint: Completion deadline (testable)
2020-08-25: Fedora Beta Freeze
- If rebuild with 3.9.0rc1 is needed, we should strive to do it before the freeze - there is a window of 2 weeks.
2020-09-14: Python 3.9.0 candidate 2
2020-09-15: Fedora 33 Beta Release (Preferred Target)
- Beta will likely be released with 3.9.0rc1.
2020-09-22: Fedora 33 Beta Target date #1
2020-10-05: Python 3.9.0 final
2020-10-06: Fedora 33 Final Freeze
- We'll update to 3.9.0 final using a freeze exception.
2020-10-20: Fedora 33 Preferred Final Target date
2020-10-27: Fedora 33 Final Target date #1

(From Python 3.9 Release Schedule and Fedora 33 Release Schedule.)

The schedule is somewhat tight for Fedora 33, but Python's annual release cycle was adapted for Fedora, hence we have a better chance of making it in time. In the future, it is expected that Python will be upgraded on a similar schedule in every odd-numbered Fedora release.

Note that upstream's "release candidates" are frozen except for blocker bugs. Since we can and will backport blocker fixes between Fedora and upstream, we essentially treat the Release Candidate as the final release.

Notes from the previous upgrade

There are notes from the previous upgrade available, so this upgrade may go smoother: SIGs/Python/UpgradingPython

Benefit to Fedora

Fedora aims to showcase the latest in free and open source software - we should have the most recent release of Python 3. Packages in Fedora can use the new features from 3.9.

There's also a benefit to the larger Python ecosystem: by building Fedora's packages against 3.9 while it's still in development, we can catch critical bugs before the final 3.9.0 release.

Scope

We will coordinate the work in a side tag and merge when ready.

Proposal owners:
1. Introduce python3.9 for all Fedoras
2. Prepare stuff in Copr as explained in description.
3. Retire python3 from F33+
4. Update python-rpm-macros so python3.9 builds python3
5. Mass rebuild all the packages that BR python3/python3-devel/... (3000+ listed in Python 3 Porting Database for Fedora)
6. Reintroduce python3.8 from Fedora 31. Update it to have all fixes and enhancements from python3 in Fedora 32 (or 33 before this change)

Other developers: Maintainers of packages that fail to rebuild during the rebuilds will be asked, using e-mail and bugzilla, to fix or remove their packages from the distribution. If any issues appear, they should be solvable either by communicating with the respective upstreams first and/or applying downstream patches. Also the package maintainers should have a look at: Porting to Python 3.9. The python-maint team will be available to help with fixing issues.

Release engineering: #9046 A targeted rebuild for all python packages will be required, before the mass rebuild.
- List of deliverables: nope

Policies and guidelines: nope

Trademark approval: nope

Upgrade/compatibility impact

All the packages that depend on Python 3 must be rebuilt. User written Python 3 scripts/applications may require a small amount of porting, but mostly Python 3.8 is forward compatible with Python 3.9.

How To Test

Interested testers do not need special hardware. If you have a favorite Python 3 script, module, or application, please test it with Python 3.9 and verify that it still works as you would expect. If the application you are testing does not require any other modules, you can test it using python3.9 even before this change is implemented, in Fedora 30, 31 or 32.

In case your application requires other modules, or if you are testing an rpm package, it is necessary to install the 3.9 version of the python3 rpm. Right now that rpm is available in copr, along with all other python packages that build successfully with python 3.9. See https://copr.fedorainfracloud.org/coprs/g/python/python3.9/ for detailed instructions how to enable Python 3.9 copr for mock.

Once the change is in place, test if you favorite Python apps are working as they were before. File bugs if they don't.

User Experience

Regular distro users shouldn't notice any change in system behavior other than the Python 3 interpreter will be in version 3.9.

Dependencies

3000+ packages depend on python3. See scope section.

Contingency Plan

Contingency mechanism: Do not merge the side tag with rawhide. If the side tag has been merged and issues arise, that will justify a downgrade, then use an epoch tag to revert to 3.8 version (never needed before)
Contingency deadline: TBD
Blocks release? Yes, we'd like to block Fedora 33 release on at least 3.9.0rc1
Blocks product? See above

Documentation

Python 3.9 Release Schedule

Features for 3.9

What's new in 3.9

Porting to Python 3.9

Release Notes

Migrating user installed packages - https://pagure.io/fedora-docs/release-notes/issue/503

@@ Line 68: / Line 68: @@
 * 2019-06-04: Python 3.9 development begins
 * 2019-11-19: Python 3.9.0 alpha 1
-** Package it as {{package|python39}} for testing purposes
+** Package it as {{package|python3.9}} for testing purposes
 ** Start the bootstrap procedure in Copr
 ** Do a mass rebuild against every future release in Copr
@@ Line 77: / Line 77: @@
 * 2020-02-26: Python 3.9.0 alpha 4
 * 2020-03-23: Python 3.9.0 alpha 5
-* 2020-04-22: Python 3.9.0 alpha 6
+* 2020-04-28: Python 3.9.0 alpha 6
 * 2020-05-18: Python 3.9.0 beta 1
 ** No new features beyond this point
 ** The ideal point when we can start rebuilding in Koji
+* 2020-05-29: Actual side tag-merge
 * 2020-05-30: Expected side tag-merge (optimistic)
-* 2020-06-08: Python 3.9.0 beta 2
+* 2020-06-09: Python 3.9.0 beta 2/3
 * 2020-06-18: Expected side tag-merge (realistic)
-* 2020-06-29: Python 3.9.0 beta 3
+* 2020-06-29: Python 3.9.0 beta 4
 * 2020-07-18: Expected side tag-merge (pessimistic)
-* 2020-07-20: Python 3.9.0 beta 4
+* 2020-07-20: Python 3.9.0 beta 5
 * 2020-07-22: Fedora 33 Mass Rebuild
-** The mass rebuild happens with fourth beta. We might need to rebuild Python packages later in exceptional case.
+** The mass rebuild happens with fifth beta. We might need to rebuild Python packages later in exceptional case.
 ** If the Koji side-tag is not merged yet at this point, we defer the change to Fedora 34.
 * 2020-08-10: Python 3.9.0 candidate 1
@@ Line 130: / Line 131: @@
 * Proposal owners:
 <!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
-*# Introduce {{package|python39}} for all Fedoras
+*# Introduce {{package|python3.9}} for all Fedoras
 *# Prepare stuff in Copr as explained in description.
 *# Retire {{package|python3}} from F33+
-*# Update {{package|python-rpm-macros}} so {{package|python39}} builds {{package|python3}}
+*# Update {{package|python-rpm-macros}} so {{package|python3.9}} builds {{package|python3}}
 *# Mass rebuild all the packages that BR {{package|python3}}/{{package|python3-devel}}/... (3000+ listed in [http://fedora.portingdb.xyz/ Python 3 Porting Database for Fedora])
-*# Reintroduce {{package|python38}} from Fedora 31. Update it to have all fixes and enhancements from {{package|python3}} in Fedora 32 (or 33 before this change)
+*# Reintroduce {{package|python3.8}} from Fedora 31. Update it to have all fixes and enhancements from {{package|python3}} in Fedora 32 (or 33 before this change)
 * Other developers: Maintainers of packages that fail to rebuild during the rebuilds will be asked, using e-mail and bugzilla, to fix or remove their packages from the distribution. If any issues appear, they should be solvable either by communicating with the respective upstreams first and/or applying downstream patches. Also the package maintainers should have a look at: [https://docs.python.org/3.9/whatsnew/3.9.html#porting-to-python-3-9 Porting to Python 3.9]. The python-maint team will be available to help with fixing issues.
@@ Line 175: / Line 176: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-Interested testers do not need special hardware. If you have a favorite Python 3 script, module, or application, please test it with Python 3.9 and verify that it still works as you would expect. If the application you are testing does not require any other modules, you can test it using {{package|python39}} even before this change is implemented, in Fedora 30, 31 or 32.
+Interested testers do not need special hardware. If you have a favorite Python 3 script, module, or application, please test it with Python 3.9 and verify that it still works as you would expect. If the application you are testing does not require any other modules, you can test it using {{package|python3.9}} even before this change is implemented, in Fedora 30, 31 or 32.
 In case your application requires other modules, or if you are testing an rpm package, it is necessary to install the 3.9 version of the python3 rpm. Right now that rpm is available in copr, along with all other python packages that build successfully with python 3.9. See https://copr.fedorainfracloud.org/coprs/g/python/python3.9/ for detailed instructions how to enable Python 3.9 copr for mock.
@@ Line 184: / Line 185: @@
 <!-- If this change proposal is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-Regular distro users shouldn't notice any change in system behavior other than the Python 3 interpreter will be in version 3.8.
+Regular distro users shouldn't notice any change in system behavior other than the Python 3 interpreter will be in version 3.9.
 == Dependencies ==

Search

Changes/Python3.9: Difference between revisions