From Fedora Project Wiki
< Changes
(→Owner) |
mNo edit summary |
||
| (49 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name. This keeps all change proposals in the same namespace --> | <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name. This keeps all change proposals in the same namespace --> | ||
| Line 29: | Line 26: | ||
== Current status == | == Current status == | ||
[[Category: | [[Category:ChangeAcceptedF34]] | ||
<!-- When your change proposal page is completed and ready for review and announcement --> | <!-- When your change proposal page is completed and ready for review and announcement --> | ||
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | ||
| Line 36: | Line 33: | ||
<!-- Select proper category, default is Self Contained Change --> | <!-- Select proper category, default is Self Contained Change --> | ||
[[Category:SelfContainedChange]] | <!--[[Category:SelfContainedChange]] --> | ||
[[Category:SystemWideChange]] | |||
* Targeted release: [[Releases/ | * Targeted release: [[Releases/34| Fedora 34 ]] | ||
* Last updated: <!-- this is an automatic macro — you don't need to change this line --> {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}} | * Last updated: <!-- this is an automatic macro — you don't need to change this line --> {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}} | ||
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page | <!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page | ||
| Line 48: | Line 45: | ||
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development | CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development | ||
--> | --> | ||
* FESCo issue: | * FESCo issue: [https://pagure.io/fesco/issue/2500 #2500] | ||
* Tracker bug: | * Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1900006 #1900006] | ||
* Release notes tracker: <will be assigned by the Wrangler> | * Release notes tracker: <will not be assigned by the Wrangler - not user-facing> | ||
== Detailed Description == | == Detailed Description == | ||
<!-- Expand on the summary, if appropriate. A couple sentences suffices to explain the goal, but the more details you can provide the better. --> | <!-- Expand on the summary, if appropriate. A couple sentences suffices to explain the goal, but the more details you can provide the better. --> | ||
===== Phase 1: Analysis ===== | |||
For this change, we will start by creating a list of all packages that have a build-time dependency on make. This will be done by analyzing spec files and also by rebuilding all packages in Fedora with make removed from the buildroot to see which packages fail to build. Once we have this list, we will remove packages that already have BuildRequires:make in their spec file, and this final list[1] will be all the packages that need to be updated in Phase 2. | |||
[1] https://github.com/tstellar/fedora-change-make-buildroot/blob/main/needs_br_make.txt | |||
===== Phase 2: Package Updates ===== | |||
Once we have the list of packages that need to be updated, we will proceed with adding BuildRequires: make to all spec files that require it. This new BuildRequires will be added to the line after the last BuildRequires in the spec file. The release number for packages will '''*not*''' be incremented for this change. | |||
We will send an announcement to the devel list one week before the planned updates to give package maintainers a chance to update their packages manually if they want. After the week has passed, the spec files will be automatically updated and changes will be pushed directly to dist-git once they are ready. | |||
During this phase, a conditional dependency on make will also be added to the redhat-rpm-config package. See explanation in the [[#Feedback|Feedback]] section for more details. | |||
===== Phase 3: Update Buildroot ===== | |||
Once package spec files have been updated, then we can proceed with removing make from the BuildRoot. This will be done by removing make from the build group in Koji and by removing make from the buildsys-build group in comps (fedora-comps). In order to avoid issues with Koschei, this change will need to be made as close as possible to the start of the mass rebuild. If possible, we will try to first remove make from the mass rebuild side-tag and then remove it from rawhide after the rebuild completes. | |||
===== Phase 4: Monitor Failures ===== | |||
Once all the changes are in place, we will continue to monitor koji builds to see if there are any failures related to this change. We expect that the analysis done in Phase 1 would give us the complete list of packages that need to be updated, but it is always possible that something will be missed. | |||
== Feedback == | == Feedback == | ||
<!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. --> | <!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. --> | ||
* Removing make from the Buildroot without rebuilding the packages first has the potential to cause mass failures in Koschei. This is because Koschei builds from the latest SRPM and not from dist-git. We can minimize this problem by removing make from the buildroot as close as possible to the mass rebuild. The proposal has been updated now to account for this issue. | |||
* gcc relies on make in order to parallelize LTO builds. Therefore, removing make from the buildroot will cause build time regressions for packages that build with gcc but do not use make. To resolve this, we will add make as a conditional dependency to redhat-rpm-config: | |||
Requires: (make if gcc) | |||
== Benefit to Fedora == | == Benefit to Fedora == | ||
Based on my research[1], Fedora Rawhide has 22,309 packages, and there are approximately 10,378 packages that either use make explicitly or fail to build when make is removed form the buildroot. This means that there are around 11,931 packages that don't need make. Removing make from the BuildRoot will reduce the network traffic, download times, and disk usage for these builds in Koji and also for users doing builds with mock. | |||
Removing make (and its dependencies) will: | |||
* Reduce the BuildRoot download size by: 7.3 MB [2]: | |||
** make: 539k | |||
** gc: 104k | |||
** guile22: 6.6M | |||
** libtool-ltdl: 36k | |||
* Reduce the BuildRoot install size by; 46 MB [2]: | |||
** make: 1.6M | |||
** gc: 229k | |||
** guile22: 44M | |||
** libtool-ltdl: 71k | |||
[1] https://github.com/tstellar/fedora-change-make-buildroot | |||
[2] Package sizes are from the x86_64 packages. | |||
<!-- What is the benefit to the distribution? Will the software we generate be improved? How will the process of creating Fedora releases be improved? | <!-- What is the benefit to the distribution? Will the software we generate be improved? How will the process of creating Fedora releases be improved? | ||
| Line 91: | Line 135: | ||
== Scope == | == Scope == | ||
* Proposal owners: | * '''Proposal owners:''' Tom Stellard | ||
<!-- What work do the feature owners have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | <!-- What work do the feature owners have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | ||
* | * '''Package Maintainers:''' Fedora package maintainers should report bugs if they think there is a problem caused by this change, but otherwise there will be no action required by them. | ||
* '''Proven Packager:''' The proposal owner will need to either apply for provenpackager status or get the help of someone with provenpackager status in order to make the spec file changes that are required. | |||
<!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | |||
<!-- What work do other developers have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | <!-- What work do other developers have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | ||
* Release engineering: [https://pagure.io/releng/ | * '''Release engineering:''' [https://pagure.io/releng/issue/9829 #9829] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)? Is a mass rebuild required? include a link to the releng issue. | <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)? Is a mass rebuild required? include a link to the releng issue. | ||
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication --> | The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication --> | ||
* Policies and guidelines: | * '''Policies and guidelines:''' The packager guidelines will need to be updated to mention that BuildRequires: make is now required for all packages that need make. <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
<!-- Do the packaging guidelines or other documents need to be updated for this feature? If so, does it need to happen before or after the implementation is done? If a FPC ticket exists, add a link here. --> | <!-- Do the packaging guidelines or other documents need to be updated for this feature? If so, does it need to happen before or after the implementation is done? If a FPC ticket exists, add a link here. --> | ||
* Trademark approval: N/A (not needed for this Change) | * '''Trademark approval:''' N/A (not needed for this Change) | ||
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. --> | <!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. --> | ||
* Alignment with Objectives: | * '''Alignment with Objectives:''' This aligns with the Fedora Minimization [https://pagure.io/minimization/issue/12 objective]. | ||
<!-- Does your proposal align with the current Fedora Objectives: https://docs.fedoraproject.org/en-US/project/objectives/ ? It's okay if it doesn't, but it's something to consider --> | <!-- Does your proposal align with the current Fedora Objectives: https://docs.fedoraproject.org/en-US/project/objectives/ ? It's okay if it doesn't, but it's something to consider --> | ||
| Line 114: | Line 161: | ||
<!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
This should not impact upgrades. | |||
== How To Test == | == How To Test == | ||
| Line 132: | Line 179: | ||
<!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
This change can be tested by rebuilding affected packages. The goal is to complete this before the mass rebuild to ensure maximum testing coverage. | |||
== User Experience == | == User Experience == | ||
| Line 145: | Line 192: | ||
- Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system. | - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system. | ||
--> | --> | ||
Fedora users will not notice any difference with this change. This will only impact Fedora package maintainers. | |||
== Dependencies == | == Dependencies == | ||
| Line 150: | Line 198: | ||
<!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
gcc < 11 will fail if the -flto=auto option is used when make is not installed. Phase 3 cannot be completed until gcc 11 lands in rawhide, but Phase 1 and Phase 2 are not blocked by this. | |||
== Contingency Plan == | == Contingency Plan == | ||
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan? This might be as simple as "Revert the shipped configuration". Or it might not (e.g. rebuilding a number of dependent packages). If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy. --> | <!-- If you cannot complete your feature by the final development freeze, what is the backup plan? This might be as simple as "Revert the shipped configuration". Or it might not (e.g. rebuilding a number of dependent packages). If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy. --> | ||
* Contingency mechanism: | * '''Contingency mechanism:''' If we discover critical issues during phase 4, then Fedora Release Engineering will need to re-add make into the buildroot. <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
<!-- When is the last time the contingency mechanism can be put in place? This will typically be the beta freeze. --> | <!-- When is the last time the contingency mechanism can be put in place? This will typically be the beta freeze. --> | ||
* Contingency deadline: | * '''Contingency deadline:''' 2021-02-23 (F34 Beta Freeze) <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? --> | <!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? --> | ||
* Blocks release? | * '''Blocks release?''' No <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
* Blocks product? | * '''Blocks product?''' No <!-- Applicable for Changes that blocks specific product release/Fedora.next --> | ||
== Documentation == | == Documentation == | ||
| Line 166: | Line 214: | ||
<!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> | ||
The packager guidelines will need to be updated to mention that BuildRequires: make is now required for all packages that need make. | |||
== Release Notes == | == Release Notes == | ||
Latest revision as of 19:19, 27 December 2020
Remove make from BuildRoot
Summary
This change will remove make from the default buildroot in Koji and mock.
Owner
- Name: Tom Stellard
- Email: <tstellar@redhat.com>
Current status
- Targeted release: Fedora 34
- Last updated: 2020-12-27
- FESCo issue: #2500
- Tracker bug: #1900006
- Release notes tracker: <will not be assigned by the Wrangler - not user-facing>
Detailed Description
Phase 1: Analysis
For this change, we will start by creating a list of all packages that have a build-time dependency on make. This will be done by analyzing spec files and also by rebuilding all packages in Fedora with make removed from the buildroot to see which packages fail to build. Once we have this list, we will remove packages that already have BuildRequires:make in their spec file, and this final list[1] will be all the packages that need to be updated in Phase 2.
[1] https://github.com/tstellar/fedora-change-make-buildroot/blob/main/needs_br_make.txt
Phase 2: Package Updates
Once we have the list of packages that need to be updated, we will proceed with adding BuildRequires: make to all spec files that require it. This new BuildRequires will be added to the line after the last BuildRequires in the spec file. The release number for packages will *not* be incremented for this change.
We will send an announcement to the devel list one week before the planned updates to give package maintainers a chance to update their packages manually if they want. After the week has passed, the spec files will be automatically updated and changes will be pushed directly to dist-git once they are ready.
During this phase, a conditional dependency on make will also be added to the redhat-rpm-config package. See explanation in the Feedback section for more details.
Phase 3: Update Buildroot
Once package spec files have been updated, then we can proceed with removing make from the BuildRoot. This will be done by removing make from the build group in Koji and by removing make from the buildsys-build group in comps (fedora-comps). In order to avoid issues with Koschei, this change will need to be made as close as possible to the start of the mass rebuild. If possible, we will try to first remove make from the mass rebuild side-tag and then remove it from rawhide after the rebuild completes.
Phase 4: Monitor Failures
Once all the changes are in place, we will continue to monitor koji builds to see if there are any failures related to this change. We expect that the analysis done in Phase 1 would give us the complete list of packages that need to be updated, but it is always possible that something will be missed.
Feedback
- Removing make from the Buildroot without rebuilding the packages first has the potential to cause mass failures in Koschei. This is because Koschei builds from the latest SRPM and not from dist-git. We can minimize this problem by removing make from the buildroot as close as possible to the mass rebuild. The proposal has been updated now to account for this issue.
- gcc relies on make in order to parallelize LTO builds. Therefore, removing make from the buildroot will cause build time regressions for packages that build with gcc but do not use make. To resolve this, we will add make as a conditional dependency to redhat-rpm-config:
Requires: (make if gcc)
Benefit to Fedora
Based on my research[1], Fedora Rawhide has 22,309 packages, and there are approximately 10,378 packages that either use make explicitly or fail to build when make is removed form the buildroot. This means that there are around 11,931 packages that don't need make. Removing make from the BuildRoot will reduce the network traffic, download times, and disk usage for these builds in Koji and also for users doing builds with mock.
Removing make (and its dependencies) will:
- Reduce the BuildRoot download size by: 7.3 MB [2]:
- make: 539k
- gc: 104k
- guile22: 6.6M
- libtool-ltdl: 36k
- Reduce the BuildRoot install size by; 46 MB [2]:
- make: 1.6M
- gc: 229k
- guile22: 44M
- libtool-ltdl: 71k
[1] https://github.com/tstellar/fedora-change-make-buildroot
[2] Package sizes are from the x86_64 packages.
Scope
- Proposal owners: Tom Stellard
- Package Maintainers: Fedora package maintainers should report bugs if they think there is a problem caused by this change, but otherwise there will be no action required by them.
- Proven Packager: The proposal owner will need to either apply for provenpackager status or get the help of someone with provenpackager status in order to make the spec file changes that are required.
- Release engineering: #9829 (a check of an impact with Release Engineering is needed)
- Policies and guidelines: The packager guidelines will need to be updated to mention that BuildRequires: make is now required for all packages that need make.
- Trademark approval: N/A (not needed for this Change)
- Alignment with Objectives: This aligns with the Fedora Minimization objective.
Upgrade/compatibility impact
This should not impact upgrades.
How To Test
This change can be tested by rebuilding affected packages. The goal is to complete this before the mass rebuild to ensure maximum testing coverage.
User Experience
Fedora users will not notice any difference with this change. This will only impact Fedora package maintainers.
Dependencies
gcc < 11 will fail if the -flto=auto option is used when make is not installed. Phase 3 cannot be completed until gcc 11 lands in rawhide, but Phase 1 and Phase 2 are not blocked by this.
Contingency Plan
- Contingency mechanism: If we discover critical issues during phase 4, then Fedora Release Engineering will need to re-add make into the buildroot.
- Contingency deadline: 2021-02-23 (F34 Beta Freeze)
- Blocks release? No
- Blocks product? No
Documentation
The packager guidelines will need to be updated to mention that BuildRequires: make is now required for all packages that need make.