DynamicBuildRequires
Summary
Add possibility to generate build-time dependencies within RPM spec file and teach RPM and mock how to handle this.
Owner
- Name: Igor Gnatenko, Florian Festi, Miroslav Suchý
- Email: ignatenkobrain@fedoraproject.org, ffesti@redhat.com, miroslav@suchy.cz
- Release notes owner:
Current status
- Targeted release: Fedora 31
- Last updated: 2019-04-12
- Tracker bug: <will be assigned by the Wrangler>
Detailed Description
For many languages (Rust, Golang, Node.Js, Ruby, Python), BuildRequires can be automatically generated. All it takes, run some special tool which will output dependencies in RPM format.
Changes in tooling
rpmbuild
- New
%generate_buildrequires
section ("script" later in text) in spec file - New option
-br
which will run script, check dynamic BuildRequires- If all of them are satisfied, generate
src.rpm
- If some of them are not satisfied or
--nodeps
is present, generatebuildreqs.nosrc.rpm
- If all of them are satisfied, generate
src.rpm
produced byrpmbuild -ba
will contain dynamic BuildRequires- Option
-bs
does not change behavior src.rpm
produced from spec file which has%generate_buildrequires
hasRequires: rpmlib(DynamicBuildRequires)
src.rpm
which contain dynamic BuildRequires hasProvides: rpmlib(DynamicBuildRequires)
for identification purposes
mock
- New option in config
dynamic_buildrequires
which controls support of new features - Support for rpmbuild changes if this option enabled
Example
Probably easiest demonstration how this works is to show example. First, let's make necessary changes in our spec file (I'm taking rust-rpick
).
diff --git a/rust-rpick.spec b/rust-rpick.spec index 716a185..01a6d35 100644 --- a/rust-rpick.spec +++ b/rust-rpick.spec @@ -16,12 +16,6 @@ Source: %{crates_source} ExclusiveArch: %{rust_arches} BuildRequires: rust-packaging -BuildRequires: (crate(dirs/default) >= 1.0.0 with crate(dirs/default) < 2.0.0) -BuildRequires: (crate(rand/default) >= 0.6.0 with crate(rand/default) < 0.7.0) -BuildRequires: (crate(serde/default) >= 1.0.0 with crate(serde/default) < 2.0.0) -BuildRequires: (crate(serde/derive) >= 1.0.0 with crate(serde/derive) < 2.0.0) -BuildRequires: (crate(serde_yaml/default) >= 0.8.0 with crate(serde_yaml/default) < 0.9.0) -BuildRequires: (crate(structopt/default) >= 0.2.0 with crate(structopt/default) < 0.3.0) %global _description \ Helps you pick items from a list by various algorithms. Example uses: pick a\ @@ -69,6 +63,9 @@ which use "default" feature of "%{crate}" crate. %autosetup -n %{crate}-%{version_no_tilde} -p1 %cargo_prep +%generate_buildrequires +%cargo_generate_buildrequires + %build %cargo_build
Then let's try to build package (I will skip parts like rustc executions because it is irrelevant).
⋊> ~/P/s/r/rust-rpick on master ⨯ fedpkg mockbuild --mock-config $PWD/fedora-rawhide-x86_64-buildreqs.cfg Wrote: /home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm INFO: mock.py version 1.4.14 starting (python version = 3.7.3)... […] INFO: Start(/home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm) Config(fedora-rawhide-x86_64-buildreqs) […] Start: build phase for rust-rpick-0.3.0-1.fc31.src.rpm Start: build setup for rust-rpick-0.3.0-1.fc31.src.rpm Building target platforms: x86_64 Building for target x86_64 Wrote: /builddir/build/SRPMS/rust-rpick-0.3.0-1.fc31.src.rpm Repository 'kkt' is missing name in configuration, using id. fedora 21 kB/s | 19 kB 00:00 kkt 2.6 MB/s | 3.0 kB 00:00 Dependencies resolved. ================================================================================ Package Arch Version Repo Size ================================================================================ Installing: rust-packaging x86_64 6-28.fc31+buildreqs kkt 13 k Installing dependencies: […] Finish: build setup for rust-rpick-0.3.0-1.fc31.src.rpm Start: rpmbuild rust-rpick-0.3.0-1.fc31.src.rpm Start: Outputting list of installed packages Finish: Outputting list of installed packages Building target platforms: x86_64 Building for target x86_64 Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.0m57Wu + umask 022 + cd /builddir/build/BUILD + cd /builddir/build/BUILD + rm -rf rpick-0.3.0 + /usr/bin/gzip -dc /builddir/build/SOURCES/rpick-0.3.0.crate + /usr/bin/tar -xof - + STATUS=0 + '[' 0 -ne 0 ']' + cd rpick-0.3.0 + /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w . + set -eu + /usr/bin/mkdir -p .cargo + cat + /usr/bin/rm -f Cargo.lock + exit 0 Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.LooeBv + umask 022 + cd /builddir/build/BUILD + cd /builddir/build/BUILD + cd rpick-0.3.0 + /usr/bin/cargo-inspector -BR Cargo.toml + /usr/bin/cargo-inspector -TR Cargo.toml + exit 0 Wrote: /builddir/build/SRPMS/rust-rpick-0.3.0-1.fc31.buildreqs.nosrc.rpm INFO: Dynamic buildrequires detected Repository 'kkt' is missing name in configuration, using id. fedora 17 kB/s | 19 kB 00:01 kkt 2.8 MB/s | 3.0 kB 00:00 Package rust-packaging-6-28.fc31+buildreqs.x86_64 is already installed. Dependencies resolved. ================================================================================ Package Arch Version Repo Size ================================================================================ Installing: rust-dirs+default-devel noarch 1.0.5-1.fc31 fedora 7.6 k rust-rand+default-devel noarch 0.6.5-3.fc31 fedora 8.4 k rust-serde+default-devel noarch 1.0.90-1.fc31 fedora 12 k rust-serde+derive-devel noarch 1.0.90-1.fc31 fedora 12 k rust-serde_yaml+default-devel noarch 0.8.8-2.fc30 fedora 8.3 k rust-structopt+default-devel noarch 0.2.15-2.fc31 fedora 8.2 k Installing dependencies: […] Building target platforms: x86_64 Building for target x86_64 Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.l2cC5s + umask 022 + cd /builddir/build/BUILD + cd /builddir/build/BUILD + rm -rf rpick-0.3.0 + /usr/bin/gzip -dc /builddir/build/SOURCES/rpick-0.3.0.crate + /usr/bin/tar -xof - + STATUS=0 + '[' 0 -ne 0 ']' + cd rpick-0.3.0 + /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w . + set -eu + /usr/bin/mkdir -p .cargo + cat + /usr/bin/rm -f Cargo.lock + exit 0 Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.cvhwuw + umask 022 + cd /builddir/build/BUILD + cd /builddir/build/BUILD + cd rpick-0.3.0 + /usr/bin/cargo-inspector -BR Cargo.toml + /usr/bin/cargo-inspector -TR Cargo.toml + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.dlI3ax + umask 022 + cd /builddir/build/BUILD + cd rpick-0.3.0 + /usr/bin/env CARGO_HOME=.cargo RUSTC_BOOTSTRAP=1 /usr/bin/cargo build -j8 -Z avoid-dev-deps --release […] Finish: build phase for rust-rpick-0.3.0-1.fc31.src.rpm INFO: Done(/home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm) Config(fedora-rawhide-x86_64-buildreqs) 2 minutes 9 seconds INFO: Results and/or logs in: /home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/results_rust-rpick/0.3.0/1.fc31 INFO: Cleaning up build root ('cleanup_on_success=True') Start: clean chroot Finish: clean chroot Finish: run
Let's check dependencies of src.rpm.
⋊> ~/P/s/r/rust-rpick on master ⨯ rpm -qpR $PWD/results_rust-rpick/0.3.0/1.fc31/rust-rpick-0.3.0-1.fc31.src.rpm (crate(dirs/default) >= 1.0.0 with crate(dirs/default) < 2.0.0) (crate(rand/default) >= 0.6.0 with crate(rand/default) < 0.7.0) (crate(serde/default) >= 1.0.0 with crate(serde/default) < 2.0.0) (crate(serde/derive) >= 1.0.0 with crate(serde/derive) < 2.0.0) (crate(serde_yaml/default) >= 0.8.0 with crate(serde_yaml/default) < 0.9.0) (crate(structopt/default) >= 0.2.0 with crate(structopt/default) < 0.3.0) rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(DynamicBuildRequires) <= 4.15.0-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(RichDependencies) <= 4.12.0-1 rust-packaging
Benefit to Fedora
Packagers won't have to pre-generate BuildRequires in the spec file which means it will be always updated (and correct) :
- Packagers can focus of making their packages better instead of spending all their packaging time copying BuildRequires from documentation and third party tools.
- BuildRequires are dropped as soon as they're no longer necessary
- Packages can be easily bumped without requiring a manual BuildRequires refresh
- BuildRequires and Requires generation can use similar utilities, making sure that the deps packages declare can also be used for second-level building. Packages no longer need to declare the deps of their second and n-th dependencies because someone forgot to declare them in the correct package.
Scope
- Proposal owners: Implement support for a feature in RPM and mock. Make use of it in interested ecosystems.
- Other developers: Maintainers of language stacks are advised to use this feature.
- Release engineering: #8129
- List of deliverables: N/A
- Policies and guidelines: Packaging Guidelines need to be updated with instructions how to use this feature.
- Trademark approval: N/A (not needed for this Change)
Upgrade/compatibility impact
Packagers and users who use repoquery might be affected (src.rpm might not contain generated dependencies).
How To Test
- Make sure you have new
rpm-build
(OPTION 1) ormock
(OPTION 2). - Add
%generate_buildrequires
script into spec file which prints dependencies, remove static BuildRequires - Build package
- OPTION1: Run
rpmbuild -ba
with your usual workflow - OPTION2: Run
mock -r fedora-rawhide-x86_64
with your usual workflow
- OPTION1: Run
- Check that your package builds fine, dependencies are getting checked/installed and final src.rpm contains generated BuildRequires
User Experience
Users won't notice differences.
Dependencies
Required feature needs to be implemented in RPM and mock.
Contingency Plan
- Contingency mechanism: (What to do? Who will do it?) Proposal Owners might still ship feature disabled for Fedora buildsystem but have it available for end-users, and move full completion to the next release.
- Contingency deadline: Beta Freeze
- Blocks release? No.
- Blocks product? No.
Documentation
TBD.
Release Notes
TBD.