From Fedora Project Wiki
(Update according to https://pagure.io/fesco/issue/1890)
Line 1: Line 1:
In the interest of keeping Fedora as a self-hosted distribution (meaning you can use Fedora version Z to build Fedora version Z from source RPMs), [[User:Mdomsch|Matt Domsch]] regularly runs a full rebuild of the "rawhide" tree, building rawhide with rawhide.  This catches a number of packages that no longer build, and need developer attention.  The results of each run are presently mailed to each failing package's owner and cc: list (as noted in the package database), and sent to fedora-devel-list.
This page describes '''the policy for packages that no longer build''' and need developer attention.


In the interest of tracking these failures, new bugs for each failing package will be filed in Bugzilla.  These bugs will all block a blocker bug, one per OS version, such as alias "F10FTBFS". Included in these bugs will be the root.log, build.log, and mock.log from mock for architectures where the failures occurred.  These bugs are automatically tagged with keyword 'Triaged', per the [[BugZappers/BugStatusWorkFlow|BugZappers workflow]].
The schedule for most releases of Fedora includes a mass rebuild (e.g. [[Fedora 27 Mass Rebuild]], [[Fedora 28 Mass Rebuild]]) to update the packages with new features of the compiler, packaging, build flags, etc. This serves as a convenient opportunity to detect all packages which no longer build properly.


A recursive check will be made that there is not already a bug that's blocking FTBFS for the package in question.  If there's not, a new bug will be filed against the package.
== Package Removal for Long-standing FTBFS bugs ==
 
Packages which fail to build will be retired after a period of time.


== Challenges ==
# Releng perfoms the '''mass rebuild''' and will open bugs for any packages which fail.
* Avoiding false positives:  It somewhat often happens that a whole class of failures are due to either build system misconfiguration, or mirrors being slightly out of sync.
# Maintainers should either fix and close the bug or acknowledge that they are working on a solution by setting the state to ASSIGNED.
* Bugs in required packages:  If glibc is broken on a particular day, it can affect a large number of package builds.  It's most appropriate to file a single bug against glibc in this case rather than many bugs against each package that hit the bug. Unfortunately, figuring this out requires human intervention.
# If a FTBFS bug remains in NEW state, a weekly reminder will be sent. The package will be '''orphaned''' after the FTBFS bug is in NEW state for 8 weeks.
# The normal [[Orphaned package that need new maintainers]] procedure will be followed for the packages orphaned in this way, leading to their retirement if nobody adopts them.
# A week before the mass branching any packages which still have open FTBFS bugs from the ''previous'' release will be '''retired'''.


# Being that this is a monthly event, I think that simple sanity checking is really all that's required here - nothing fancy.  Rebuild and filing should be two separate phases, so that these issues can be caught.
(Effectively, packages will be retired after 14 weeks if there is no maintainer response and the package is orphaned, or after 6½ months if the maintainer responds but the bug is not fixed.)


== Process ==
Only one FTBFS bug should be open at any time against a given package. If the package FTBFS in more than one mass rebuild, the bug will be set as blocking for all relevant tracking bugs.
* File bugs, once for each package.
* Block FTBFS
* FTBFS blocks Target tracker for next release
* Attach root.log and build.log, and mock.log from each architecture that has failed.
* Fedora version = 'rawhide'
* Run monthly


== Results ==
== Tracking bugs ==
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=440169 generic FTBFS blocker bug]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=440169 generic FTBFS blocker bug]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=440169&hide_resolved=1 F9FTBFS]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=440169&hide_resolved=1 F9FTBFS]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=463452&hide_resolved=1 F10FTBFS]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=463452&hide_resolved=1 F10FTBFS]
* F11 was handled by Release Engineering, no FTBFS bugs were filed.  http://jkeating.fedorapeople.org/needed-f11-rebuilds.html
* F11 was handled by Release Engineering, no FTBFS bugs were filed.  http://jkeating.fedorapeople.org/needed-f11-rebuilds.html ([[Fedora 11 Mass Rebuild]])
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=511348&hide_resolved=1 F12FTBFS]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=511348&hide_resolved=1 F12FTBFS] ([[Fedora 12 Mass Rebuild]])
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=538681&hide_resolved=1 F13FTBFS]
* [https://bugzilla.redhat.com/showdependencytree.cgi?id=538681&hide_resolved=1 F13FTBFS]
** [https://bugzilla.redhat.com/showdependencytree.cgi?id=564245&hide_resolved=1 F13FTBFS due to] [[Features/ChangeInImplicitDSOLinking]]
** [https://bugzilla.redhat.com/showdependencytree.cgi?id=564245&hide_resolved=1 F13FTBFS due to] [[Features/ChangeInImplicitDSOLinking]]
* [https://bugzilla.redhat.com/show_bug.cgi?id=596849 F14FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=596849 F14FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=659965 F15FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=659965 F15FTBFS] ([[Fedora 15 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=713919 F16FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=713919 F16FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=817273 F17FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=817273 F17FTBFS] ([[Fedora 17 Mass Rebuild]])
* There seem to be no FTBFS bugs reported for Fedora 18, list of failed builds from the mass rebuild: http://fedorapeople.org/~ausil/f18-failures.html
* There seem to be no FTBFS bugs reported for Fedora 18, list of failed builds from the mass rebuild: http://fedorapeople.org/~ausil/f18-failures.html ([[Fedora 18 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=913825 F19FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=913825 F19FTBFS] ([[Fedora 19 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=991858 F20FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=991858 F20FTBFS] ([[Fedora 20 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=1105908 F21FTBFS]
* [https://bugzilla.redhat.com/show_bug.cgi?id=1105908 F21FTBFS] ([[Fedora 21 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=1305208 F24FTBFS]
* F22?
* F23?  ([[Fedora 23 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=1305208 F24FTBFS] ([[Fedora 24 Mass Rebuild]])
* No mass rebuild in F25
* [https://bugzilla.redhat.com/show_bug.cgi?id=1423041 F26FTBFS] ([[Fedora 26 Mass Rebuild]])
* F27? ([[Fedora 27 Mass Rebuild]])
* [https://bugzilla.redhat.com/show_bug.cgi?id=1555378 F28FTBFS] ([[Fedora 28 Mass Rebuild]])


== What to do if you get a FTBFS bug? ==
== What to do if you get a FTBFS bug? ==
*  Read the logs.
*  Read the logs. Each FTBFS bug should have the build logs attached.
* If the bug is in your package:
* If the bug is in your package:
# In your package's devel/ branch in CVS, fix the problems uncovered.
# Fix the problems uncovered and commit the changes.
# Commit the changes and build the package.   The fixed package will land in rawhide, generally the next day.
# Build the package. The fixed package will land in rawhide, generally the next day. If branching has already occurred, also fix the build in branched.
# If the build succeeds, close the bug as CLOSED: RAWHIDE, and include the package version number in the comments.
# If the build succeeds, close the bug as CLOSED: RAWHIDE, and include the package version number in the comments.
# If branching already occurred, but bodhi hasn't been activated yet, also build the package in branched.
# If bodhi has already been activated for branched, also make an update. An update should be made even if branched has already been released.
* If the bug is in a different package:
* If the bug is in a different package:
# Find an existing bug for that package describing the problem.  Set your bug to "Depends on" that other bug.  Do not change the component of your bug to the other package, or you will get more FTBFS bugs created against you.
# Find an existing bug for that package describing the problem.  Set your bug to "Depends on" that other bug.  Do not change the component of your bug to the other package, or you will get more FTBFS bugs created against you.
# When that other bug is closed, you'll get an email from bugzilla as usual.  Rebuild your package using a koji scratch build, to verify it builds cleanly again. Close your bug as CLOSED, RAWHIDE, mentioning this.
# When that other bug is closed, you'll get an email from bugzilla as usual.  Rebuild your package using a koji scratch build, to verify it builds cleanly again. Proceed according to points 2-5 above.
*  If the package should be retired, see [[Retired packages]]
*  If the package should be retired, see [[How to remove a package at end of life]].
 
In all cases, if you close an FTBFS bug as a duplicate of another bug, please make the other bug to block 'FTBFS' (bug number 440169).  In this way the bug left open will appear in the FTBFS reports properly.


== Package Removal for Long-standing FTBFS bugs ==
In all cases, if you close an FTBFS bug as a duplicate of another bug, please make the other bug to block the right FTBFS tracking bugs. This way the bug left open will appear in the FTBFS reports properly.


Packages with unresolved FTBFS bugs at Feature Freeze for the N+2 release will be removed from the distribution. Package owners may request that their package _not_ be removed provided they are actively working on resolving the FTBFS and have a plan to resolve the FTBFS before the N+2 Alpha release. See [[Deprecate FTBFS packages]] for more details.
== See also ==
* FESCo ticket updating this policy: https://pagure.io/fesco/issue/1890
* releng docs: https://docs.pagure.org/releng/sop_deprecate_ftbfs_packages.html

Revision as of 21:21, 21 May 2018

This page describes the policy for packages that no longer build and need developer attention.

The schedule for most releases of Fedora includes a mass rebuild (e.g. Fedora 27 Mass Rebuild, Fedora 28 Mass Rebuild) to update the packages with new features of the compiler, packaging, build flags, etc. This serves as a convenient opportunity to detect all packages which no longer build properly.

Package Removal for Long-standing FTBFS bugs

Packages which fail to build will be retired after a period of time.

  1. Releng perfoms the mass rebuild and will open bugs for any packages which fail.
  2. Maintainers should either fix and close the bug or acknowledge that they are working on a solution by setting the state to ASSIGNED.
  3. If a FTBFS bug remains in NEW state, a weekly reminder will be sent. The package will be orphaned after the FTBFS bug is in NEW state for 8 weeks.
  4. The normal Orphaned package that need new maintainers procedure will be followed for the packages orphaned in this way, leading to their retirement if nobody adopts them.
  5. A week before the mass branching any packages which still have open FTBFS bugs from the previous release will be retired.

(Effectively, packages will be retired after 14 weeks if there is no maintainer response and the package is orphaned, or after 6½ months if the maintainer responds but the bug is not fixed.)

Only one FTBFS bug should be open at any time against a given package. If the package FTBFS in more than one mass rebuild, the bug will be set as blocking for all relevant tracking bugs.

Tracking bugs

What to do if you get a FTBFS bug?

  • Read the logs. Each FTBFS bug should have the build logs attached.
  • If the bug is in your package:
  1. Fix the problems uncovered and commit the changes.
  2. Build the package. The fixed package will land in rawhide, generally the next day. If branching has already occurred, also fix the build in branched.
  3. If the build succeeds, close the bug as CLOSED: RAWHIDE, and include the package version number in the comments.
  4. If branching already occurred, but bodhi hasn't been activated yet, also build the package in branched.
  5. If bodhi has already been activated for branched, also make an update. An update should be made even if branched has already been released.
  • If the bug is in a different package:
  1. Find an existing bug for that package describing the problem. Set your bug to "Depends on" that other bug. Do not change the component of your bug to the other package, or you will get more FTBFS bugs created against you.
  2. When that other bug is closed, you'll get an email from bugzilla as usual. Rebuild your package using a koji scratch build, to verify it builds cleanly again. Proceed according to points 2-5 above.

In all cases, if you close an FTBFS bug as a duplicate of another bug, please make the other bug to block the right FTBFS tracking bugs. This way the bug left open will appear in the FTBFS reports properly.

See also

Retrieved from "https://fedoraproject.org/w/index.php?title=Fails_to_build_from_source&oldid=518858"