Changes/rpmautospec

rpmautospec - removing release and changelog fields from spec files

Summary

The goal of this change is to deploy in production the rpmautospec project.

With it, the content of the Release and %changelog fields in spec files can be auto-generated, either locally or in the builder using the information present in the git repo (in the form of git tags).

Note: This proposal is about changing the way the Release and %changelog sections of the spec files are filled, not about removing them from the SRPM or binary RPM.

Owner

• Name: Pierre-Yves Chibon, Nils Philippsen
• Email: pingou - at - pingoured.fr, nphilipp - at - redhat.com

Current status

• Targeted release: Q3 2021 ( Fedora 35 )
• Last updated: 2021-06-18
• FESCo issue: #2582
• Tracker bug: #1945406
• Release notes tracker: <will NOT be assigned by the Wrangler>

Detailed Description

rpmautospec offers packagers who want to use it the possibility of replacing the content of the Release of their spec file by Release: %autorelease and/or replace the content of the %changelog section of their spec file by:

 %changelog
 %autochangelog

Both %autorelease and %autochangelog are RPM macros that will be expanded or replaced when the SRPM is built on the build system by their corresponding values according to rpmautospec.

An overview of how rpmautospec works can be found at: https://docs.pagure.org/fedora-infra.rpmautospec/principle.html. We will describe below how each macro works.

The %autorelease macro

To determine the next release information, rpmautospec relies on the git history of the package. This information is extracted from the package repository when run as a koji plugin or locally (e.g. using fedpkg).

Using the git history of the package as well as flags set by the packager in the spec file, rpmautospec then computes the next release value for the package.

Once defined, it prepends a suitably defined %autorelease macro to the top of the spec file, freezing the computed value of the release number and thus allowing reproducible builds of the SRPM.

The documentation of the autorelease macro describes how packagers can use it to provide extra information.

The %autochangelog macro

The %autochangelog macro is in fact more a placeholder that rpmautospec fills in during the creation of the SRPM (or when ran manually on a project).

rpmautospec uses two sources of information to automatically generate the changelog:

• an optional changelog file that packagers can add to the repository
• the git history of the spec file

rpmautospec will include the changelog as is in the %changelog section and will use all the commits made to the repository after the last change of the changelog file to generate the changelog.

In other words, if the packager has a repository created on January 10th and works on it for 6 months, adds a changelog file on June 1st. All the commits made before that commit of June 1st will be ignored in favor of the content of the changelog file. Similarly, if the packager then edits the file on July 1st, all the commits prior to that commit of July 1st will be ignored. All the commits involving files ending with either ".spec" or ".patch" (this list can be expended if desired) and made after July 1st will be used to generate the changelog.

Benefit to Fedora

In short: easier packaging in Fedora for the packagers who opt-in.

The Release and %changelog fields are the two most conflicting fields in RPM spec files. They impact most pull requests if they involve updating the package or if the package is updated/rebuilt while pull-request are being reviewed.

We currently have three different change logs per package and while they target different audiences (package maintainer: git, sysadmin: %changelog and end-user: bodhi notes) they overlap a lot and in most cases are redundant. With this proposal, package maintainers will only have to cope with two changelogs: git and bodhi notes.

Scope

• Proposal owners:
  • Finish the work on the %autorelease macro to support some of the use cases not implemented currently and change the algorithm to only consider the number of commits since the last version change (as requested by FESCo)
  • Adjust rpmdev-bumpspec (used by releng for mass-rebuilds) so it ignores spec files using %autorelease/%autochangelog
  • Adjust the mass-rebuild script so it only adds a suitable git commit log message for spec files using %autorelease/%autochangelog (NB: uses rpmdev-bumpspec, so no change necessary here)
  • Adjust fedpkg to skip the NEVR check when using rpmautospec
  • Add dependency on rpmautospec on redhat-rpm-config

• Other developers:
  • Opt-in for those who want to use it
  • Test changes in staging
  • Provide feedback

• Release engineering: #Releng issue 10011

• Policies and guidelines: Packaging guidelines should be adjust to explain how rpmautospec works and can be used. Documentation at: https://docs.pagure.org/fedora-infra.rpmautospec/ should provide a good basis for this

• Trademark approval: N/A (not needed for this Change)

• Alignment with Objectives: While not listed on the wiki page of that objective, it could be seen as related to https://fedoraproject.org/wiki/Objectives/Packager_Experience

Upgrade/compatibility impact

Not applicable

How To Test

In staging:

• Use stg-koji instead of koji
• Use fedpkg-stage instead of fedpkg (dnf install fedpkg-stage)
• Run kinit <accountname>@STG.FEDORAPROJECT.ORG
• Test with rawhide as rpmautospec is only available there in staging at this point
• Install rpmautospec-rpm-macros to build packages locally.
• Edit the spec file to use either macro (or both)
• (Optional) Run rpmautospec calculate-release/generate-changelog/process-distgit on the git repository of your choice to see what it does.
• Build the package (fedpkg-stage build --skip-nvr-check until changes have landed in rpkg/fedpkg)
• Check its release or changelog according to the macro set

In production:

• Edit the spec file to use either macro (or both)
• (Optional) Run rpmautospec on the git repository of your choice to see what it does (you may have to run rpmautospec tag-package the first time you run it).
• Build the package
• Check its release or changelog according to the macro set

Issues can be reported at: https://pagure.io/fedora-infra/rpmautospec/issues

Note that using this approach has some peculiarities, we've documented the ones we've encountered at: https://docs.pagure.org/fedora-infra.rpmautospec/peculiarities.html

User Experience

Packagers not opting-in should not be affected in any way by this change.

Packagers opting-in should be able to stop worrying (or worry much less) about the content of the Release/%changelog fields in their spec file.

Dependencies

None

Contingency Plan

• Contingency mechanism: rpmautospec is not deployed in koji
• Contingency deadline: N/A (this isn't tied to a Fedora release)
• Blocks release? No

Documentation

All the upstream documentations can be https://docs.pagure.org/fedora-infra.rpmautospec/

Release Notes

N/A, this change will not affect end-users (except maybe for some changes in the content of the rpm changelog).