Fedora 29 Mass Rebuild

Goal

The goal is to rebuild every single Fedora package, regardless of content, before the Fedora 29 Beta Change Deadline.

Driving Features

• GLIBC228
• BINUTILS230
• i686_Is_For_x86-64
• BINUTILS231

Schedule

Given the changes required for the above features and the given schedules, Release Engineering will be ready to start scripted rebuilds on Wednesday, July 11th. All automated rebuilds should be finished prior to the Feature Freeze. Any clean-up manual rebuilds should be finished prior to the Beta Change Deadline.

NOTE: As BINUTILS231 requires more time to test and fix issues, we are postponing the mass rebuild and we will re-evaluate the status on Friday, July 13th.

NOTE: Issues with BINUTILS231 got fixed with binutils-2.30.90-3.fc29 build, so we are starting mass rebuilds early on Thursday, July 12th.

Scripts

Release Engineering has created two scripts. One is to initiate the builds, and the other is to query for items still needing to be built.

Build Initiation

The rebuild script works in the following way:

 Generate a list of all packages in f29
 Loop through each package:
   Query if a build has already been attempted/completed since koji changes went live.
   If so, move to next package
   If not, check out git
   rpmdev-bumpspec
   fedpkg commit -cp
   fedpkg (background) build
   Move on to next package

Each step will have some error catching and logging so that people can clean up the various failures for whatever reasons. The builds will be background builds, which will allow other builds done to take higher priority when a builder has a free slot. However maintainers should take care when doing this so that the background build won't take 'latest' precedent when it finishes. Asking rel-eng to kill the scripted build will typically be enough.

Build Tagging

Once the rebuild script has finished, another script will run to tag the builds into f29. This script will check that a newer build hasn't been done or started in f29 since the scripted rebuild was submitted. This will protect against the scripted rebuild overriding an even newer build done while the scripted rebuild was waiting in the background.

Status Query

Release Engineering has developed a script to query f29 and report on packages that have yet to be rebuilt. Results of these queries will be delivered to various places, yet to be determined, broken down by maintainer for easier discovery of work needed.

• Packages that failed to build
• Packages that need to be rebuilt

Maintainer Actions

• Maintainers can help this effort by ensuring rpmdev-bumpspec correctly bumps your package's spec files.
• Maintainers should ensure that their packages currently build from source.
• Maintainers should ensure that there are no unwanted changes committed to git but not built yet.

Frequently Asked Questions

When will my own build "count" for the rebuild?

After the last piece needed for the mass rebuild a rpm build was put into place, a build done by a maintainer will "count" toward the rebuild.

Will the rebuilds be ordered?

Currently there is no plan to logically order the rebuilds based on deps. The ordering will be alphanumerical based on package name. There are no planned ABI/soname changes that would require logical build ordering. Further, all the builds will be kept in a special tag (f29-rebuild) until the whole run is done, and then they will be tagged into f29 in one shot to minimize the shuffle of buildroot contents during the rebuilds.

Feedback

Questions/comments/concerns should be directed to [1], or #fedora-releng on freenode IRC.