BinutilsRawhideSync

Rawhide synchronization for the GNU Binutils

This page documents the process by which the rawhide binutils package is synced with the upstream GNU Binutils project. The aim of this process is to make syncing simple and easy so that it can be performed on a regular basis. Due to the fact that the binutils is an important component in the toolchain used to build many of the packages in rawhide however, it is important to extensively test a new binutils before putting it into the buildroot. This does make the syncing process more time consuming and complicated.

This process is based upon a similar process for the GNU C Library (GlibcRawhideSync), but tweaked to fit the binutils' way of handling patches.

This process assumes that you already have a clone of the Fedora rawhide binutils.

The decision for when to perform a sync is a complicated one. In general if there is a newly made official upstream release then that should be used. Otherwise if there is a recent upstream branch snapshot available then that should be used. Otherwise if there has not been a sync in the last two weeks then it is probably time for this procedure to be run.

Follow these steps:

1. Check to see if there is a recent snapshot available:

 https://snapshots.sourceware.org/binutils/trunk/

If there is, then download it and proceed to step 2. Otherwise proceed to step 5.

2. Go to your clone of the fedora binutils package and make sure that it is clean and up to date:

cd <rawhide-binutils>
git switch rawhide
git pull
git clean -fdx
git status 

3. Delete the current source tarball and then copy in the source tarball downloaded in step 1:

rm -f *.tar.xz
cp ~/Downloads/binutils-*.tar.xz . 

4. View the binutils.spec file and check that the source variable is set to snapshot:

# %%define source official-release
# %%define source pre-release
%define source snapshot
# %%define source tarball

Then set the commit_id variable to the commit id of the new tarball:

sed --in-place -e 's/define commit_id.*/define commit_id %(echo "COMMI_ID")/' binutils.spec 

Replace COMMIT_ID in the above command with the commit id of the tarball.

Also increment the Release: field by one and add a changelog entry for the update:

rpmdev-bumpspec -c "- Rebase to commit COMMIT_ID" binutils.spec

Replace COMMIT_ID in the above command with the commit id of the tarball.

Goto step 10.

5. (This step is reached from step 1 if a suitable upstream source snapshot is unavailable).

Make sure that your clone of the upstream GNU Binutils is up to date, on the right branch and clean. If there any problems or uncommitted changes, resolve them before continuing.

cd <upstream-binutils>
git switch HEAD
git pull
git clean -fdx
git status

Delete any old release tarballs and then create a source tarball from the upstream sources. Change the version number in the tarball's name to a commit id.

rm -f *.xz
export COMMIT_ID=`git rev-parse HEAD`
./src-release.sh -x -r `git log -1 --format=%cd --date=format:%F $COMMIT_ID` binutils
mv binutils-*.tar.xz binutils-$COMMIT_ID.tar.xz 

6. Go to your clone of the fedora binutils package and make sure that it is clean and up to date:

pushd <rawhide-binutils>
git switch rawhide
git pull
git clean -fdx
git status 

7. Delete the current source tarball and then copy in the source tarball created in step 5:

rm -f *.tar.xz
cp <upstream-binutils>/binutils*.tar.xz . 

8. [Optional] Tidy up the upstream repository.

pushd <upstream-binutils>
git clean -fdx
popd

9. Edit the binutils.spec file and change the Source0: field to reference the new tarball:

sed --in-place -e "s/Source0:.*/Source0: `ls *.tar.xz`/" binutils.spec  

Also check that the use_commit_id_tarballs define in the spec file is set to 1.

Also check that the use_snapshot_tarballs define is set to 0.

Also increment the Release: field by one and add a changelog entry for the update:

rpmdev-bumpspec -c "- Rebase to commit $COMMIT_ID" binutils.spec

10. Create a source tree and discover if any local patches no longer apply:

fedpkg prep 

If a patch does not apply, decide if it is no longer necessary, or if it needs to be adjusted. Take whatever actions are necessary to resolve this issue and then repeat step 5 until a clean run is obtained.

If a patch is not needed, remove its entry from the binutils.spec file.

Also schedule the patch to be deleted from the repository:

git rm <unneeded-patch>

Optionally add a mention of this to the changelog entry created in step 5.

- Retire: <unneeded-patch>

Optionally renumber the remaining patches referenced in the binutils.spec file so that they are in a monotonically increasing order.

11. Run a local build to ensure that there are no surprises.

 fedpkg local 

If there are problems, fix them. Note, local builds run in parallel, so it is not always easy to discover the location of a build failure. If the failure is due to a bug in the upstream sources, then it may be better to abort this sync and wait for the bug to be fixed. Or to fix the bug upstream first and then start the sync process again. If the failure is due to the local patches that are applied to the sources then they may need to be adjusted. If a failure is due to a testsuite problem then either the test might need to be tweaked, or just skipped if there is no easy way to correct it.

12. Then run a scratch build.

rm -f *.rpm
fedpkg srpm
fedpkg scratch-build --srpm *.rpm 

If the build fails, investigate and fix the problems. The most likely cause at this point is new failures in one of the testsuites. Note - the build.log file created for the build for each architecture will include a uuencoded compressed tarball of the logs from the testsuite runs. Hence the logs can be downloaded and examined locally by:

[download build.log file]
uudecode build.log
tar xvf binutils-*.tar.xz

13. [Optional] This next step is optional, but recommended. It assumes that you have command line access to the COPR system and you have the mass-prebuild package installed. Warning - this step will take a long time to complete, as in 3-4 days...

Set up a config script for the mass pre-build tool.

archs: x86_64 aarch64 ppc64le s390x
chroot: fedora-rawhide
name: binutils-$COMMIT_ID
packages:
  binutils:
    src_type: file
    src: <path-to-source-rpm-created-in-step-8>
data: <path-to-results-directory>
verbose: 1

Then use it to run a test of the new binutils

mpb --config <config-file>

If there are problems, try to find out what went wrong.

14. Finally check that the fedora repository has not received any other commits, add the new sources, and commit the changes.

git pull
fedpkg new-sources binutils-*.tar.xz binutils-2.19.50.0.1-output-format.sed
fedpkg commit -m "Rebase to commit $COMMIT_ID"
fedpkg push
fedpkg build

15. Sit back and relax. Have a cup of tea. Then wait for the bug reports to flood in.