(explicitly mark as draft) |
(Stop making the parsing script sad) |
||
(8 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
= Mass Retire Golang Leaves = | |||
= | |||
== Summary == | == Summary == | ||
As of Jan 2023, 275/1660 (17%) library only Go source packages are | As of Jan 2023, 275/1660 (17%) library only Go source packages are leaves. Overall, these packages are maintained by 35 different maintainers along with the Go SIG. [https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves These leaves]([https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves-by-maintainer by maintainer]) will be mass retired in Fedora 39. | ||
leaves. | |||
Overall, these packages are maintained by 35 different maintainers along with | |||
the Go SIG. | |||
[https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves These leaves] will be | |||
mass retired in Fedora 39. | |||
== Owner == | == Owner == | ||
Line 23: | Line 14: | ||
== Current status == | == Current status == | ||
[[Category: | [[Category:ChangeAcceptedF39]] | ||
<!-- When your change proposal page is completed and ready for review and announcement --> | <!-- When your change proposal page is completed and ready for review and announcement --> | ||
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | ||
Line 41: | Line 32: | ||
ON_QA -> change is fully code complete | ON_QA -> change is fully code complete | ||
--> | --> | ||
* FESCo issue: | * [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/GXDLFVXDD53SAR262NMSGMFJBNZ42I5C/ devel thread] | ||
* Tracker bug: | * FESCo issue: [https://pagure.io/fesco/issue/2954 #2954] | ||
* Release notes tracker: | * Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=2170956 #2170956] | ||
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/967 #967] | |||
== Definitions == | == Definitions == | ||
Line 59: | Line 51: | ||
We'd like to clean up these unneeded packages to allow us to better direct our | We'd like to clean up these unneeded packages to allow us to better direct our | ||
time and unclutter the distribution. Handling this large number of leaves one | time and unclutter the distribution. Handling this large number of leaves one | ||
by one and maintainer by maintainer proved | by one and maintainer by maintainer proved ineffective, so we're taking a more | ||
heavy handed approach and mass retiring these leaf packages prior to the | heavy handed approach and mass retiring these leaf packages prior to the | ||
release of Fedora 39. | release of Fedora 39. | ||
Line 69: | Line 61: | ||
In short: | In short: | ||
* We wrote a script to identify the ''leaf'' packages. | * We wrote a script to identify the ''leaf'' packages. | ||
* The current list of ''leaf'' packages is available | * The current list of ''leaf'' packages is available at https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves. A by maintainer breakdown is available at https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves-by-maintainer. | ||
* We will contact the maintainers of these packages and allow them to opt-out. | * We will contact the maintainers of these packages and allow them to opt-out. | ||
* We will use Copr to verify our work before proceeding with the mass removal. | * We will use Copr to verify our work before proceeding with the mass removal. | ||
Line 123: | Line 115: | ||
** ✅ Store an updated list of leaves in https://pagure.io/GoSIG/go-leaves | ** ✅ Store an updated list of leaves in https://pagure.io/GoSIG/go-leaves | ||
** Directly contact affected maintainers and devel-announce. Ensure opt outs are properly accounted for. | ** Directly contact affected maintainers and devel-announce. Ensure opt outs are properly accounted for. | ||
** Simulate the removal of these packages and rebuild the Golang stack in Copr. Investigate any new failures | ** Simulate the removal of these packages and rebuild the Golang stack in Copr. Investigate any new failures. | ||
** Mass retire packages after waiting a few weeks for maintainers to opt out. Aim to have this completed before the mass rebuild. | ** Mass retire packages after waiting a few weeks for maintainers to opt out. Aim to have this completed before the mass rebuild. | ||
* Release engineering: [https://pagure.io/releng/ | * Release engineering: [https://pagure.io/releng/issue/11253 #11253] <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
** After we preform the mass retirement, releng will allow any Go SIG member to unretire a ''leaf'' within eight weeks even if they're not the main admin. | ** After we preform the mass retirement, releng will allow any Go SIG member to unretire a ''leaf'' within eight weeks even if they're not the main admin. | ||
** The change owners will mass retire the packages themselves using go-sig/provenpackager membership. | ** The change owners will mass retire the packages themselves using go-sig/provenpackager membership. | ||
Line 152: | Line 144: | ||
The current draft of the package leaves script | The current draft of the package leaves script | ||
[https://git.sr.ht/~gotmax23/fedora-scripts/tree/main/item/go-sig/go_leaves.py is available]. | [https://git.sr.ht/~gotmax23/fedora-scripts/tree/main/item/go-sig/go_leaves.py is available]. | ||
Latest revision as of 13:33, 11 May 2023
Mass Retire Golang Leaves
Summary
As of Jan 2023, 275/1660 (17%) library only Go source packages are leaves. Overall, these packages are maintained by 35 different maintainers along with the Go SIG. These leaves(by maintainer) will be mass retired in Fedora 39.
Owner
- Name: Maxwell G; Alejandro Sáez Morollón; the Go SIG
- Email: gotmax@e.email; asm@redhat.com; golang@lists.fedoraproject.org
Current status
- Targeted release: Fedora Linux 39
- Last updated: 2023-05-11
- devel thread
- FESCo issue: #2954
- Tracker bug: #2170956
- Release notes tracker: #967
Definitions
- library only source package -- a component/source package that only contains noarch golang *-devel subpackages.
- binary subpackage -- an arched subpackage that contains go binaries.
- source package with binaries - a component/source package that contains at least one binary subpackage; may also contain library subpackages.
- leaf -- a library only source package whose subpackages don't have any dependents
Detailed Description
The Go SIG maintains over 2000 source packages. 275 of these packages are leaves. Overall, these packages are assigned to 35 different maintainers. We'd like to clean up these unneeded packages to allow us to better direct our time and unclutter the distribution. Handling this large number of leaves one by one and maintainer by maintainer proved ineffective, so we're taking a more heavy handed approach and mass retiring these leaf packages prior to the release of Fedora 39.
Source packages with binaries will be excluded even if the binary subpackage(s) and/or the -devel subpackage(s) are leaves.
See the next section for more information on how we plan to implement this. In short:
- We wrote a script to identify the leaf packages.
- The current list of leaf packages is available at https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves. A by maintainer breakdown is available at https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves-by-maintainer.
- We will contact the maintainers of these packages and allow them to opt-out.
- We will use Copr to verify our work before proceeding with the mass removal.
Implementation
Finding the leaves:
- Find the source packages that BuildRequire go-rpm-macros or any golang.src subpackage
- Iterate over the source packages list and find the subpackages for each one
- If all of a source package's subpackages are noarch and contain "-devel", it is considered a leaf.
- Remove packages from the leaves list if they've opted out.
- Store the output in https://pagure.io/GoSIG/go-leaves/blob/main/f/leaves. Update it periodically.
Confirmation:
- Create a blank blocker package that Conflicts with the to be removed packages.
- Create a new Copr with the blocker package in its default buildroot. This will simulate the actual removal of these packages.
- Rebuild all of the non leaf packages.
- Investigate new failures and rebuild in a control Copr.
Opt outs:
- A list of opt outs will be kept in https://pagure.io/GoSIG/go-leaves. Go SIG members will have
commit
. - As explained in the aforementioned repo's README, packagers can simply push an
opt-out/{component}
text file with a short justification to opt out.
Mass Retirement:
- Shallow clone the leaves' distgit repositories and run
fedpkg retire
on each. - Double check that the packages are properly blocked/retired in Koji and PDC.
Feedback
This was briefly discussed in a Go SIG meeting (minutes). Feedback was generally positive.
Benefit to Fedora
This change will allow the Go SIG to focus its limited people power on maintaining packages that are actually needed. Fedora won't waste resources uselessly rebuilding these packages. Hopefully, the other packages will be better maintained.
Scope
- Proposal owners:
- ✅ Write a script to determine leaves. (go_leaves.py)
- ✅ Store an updated list of leaves in https://pagure.io/GoSIG/go-leaves
- Directly contact affected maintainers and devel-announce. Ensure opt outs are properly accounted for.
- Simulate the removal of these packages and rebuild the Golang stack in Copr. Investigate any new failures.
- Mass retire packages after waiting a few weeks for maintainers to opt out. Aim to have this completed before the mass rebuild.
- Release engineering: #11253
- After we preform the mass retirement, releng will allow any Go SIG member to unretire a leaf within eight weeks even if they're not the main admin.
- The change owners will mass retire the packages themselves using go-sig/provenpackager membership.
- Ensure that the packages are properly blocked/retired in Koji and PDC.
Upgrade/compatibility impact
There should be none. These library packages are not meant for end users. We will not Obsolete these packages, so they won't be removed from existing systems.
Contingency Plan
The change can be deferred if the prep work isn't completed in time.
The change owners are taking multiple measures to avoid retiring packages that are not actually leaves (i.e. false positives). In the event of unforeseen issues, some or all of the packages can be unretired and rebuilt. Retagging builds from F38 is also possible, as Golang library packages only contain source code and are architecture independent.
Documentation
The current draft of the package leaves script is available.
Release Notes
Remove unused Golang libraries from the repositories. These packages are not meant for end user consumption to begin with. We will not Obsolete these packages, so they won't be removed from existing systems.