Latest revision as of 13:15, 6 May 2021

Broken RPATH will fail rpmbuild

Summary

Enable broken RPATH detection buildroot policy script by default. This will make the RPM build fail once a broken RPATH was detected within a binary or a shared library file. An opt-out mechanism will be provided as well.

Owner

Name: Charalampos Stratakis
Email: cstratak AT redhat.com

Current status

Targeted release: Fedora Linux 35
Last updated: 2021-05-06
FESCo issue: <will be assigned by the Wrangler>
Tracker bug: <will be assigned by the Wrangler>
Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

The dynamic linker and loader (ld.so) is responsible for resolving runtime dependencies of executables and shared library files through a search hierarchy. However some packages (usually through their upstream buildsystems) contain a hard-coded path within their binaries or .so files, by using the -R or -rpath flag during compilation, which is called an RPATH. By utilizing RPATH, ELF files can point to directories to be included in the search path, on runtime, to resolve their dependencies.

While RPATH can be used for non-standard directories, such as a place containing private libraries of the project, when it points to a value already provided by the search path of ld.so, it changes the hierarchy of the search by placing the system defaults first.

(a) DT_RPATH -> (b) LD_LIBRARY_PATH -> (c) DT_RUNPATH -> (d) cache (/etc/ld.so.cache) -> (e) system defaults

This could present a variety of issues, such as LD_LIBRARY_PATH overrides not working, incomplete dependency resolution, loading of wrong libraries etc. In general, changing the default search hierarchy could lead to unforeseen bugs and issues - in a similar manner as adding /usr/lib64 to LD_LIBRARY_PATH.

Another problem of a hardcoded RPATH is security. When an ELF object contains an RPATH pointed to a directory not managed by the system, where some malicious actor has write permissions to, it's relatively easy to execute arbitrary code.

Performance can be also affected, since probing explicitly e.g. /usr/lib64 through RPATH adds extra open/openat system calls to the process startup.

In Fedora the use of such RPATH is forbidden, but it was never enforced. This change intends to ratify that by executing /usr/lib/rpm/check-rpaths during rpmbuild, after %install, and fail the build if an RPATH entry was detected. This change will not affect RPATH's pointing to private libraries.

Definition of a broken RPATH

This change will use the rpm script for checking the broken RPATH's.

The categories are:

standard RPATHs (e.g. /usr/lib or /usr/lib64); such RPATHs are a minor issue but are introducing redundant searchpaths without providing a benefit. They can also cause errors in multilib environments.
invalid RPATHs; these are RPATHs which are neither absolute nor relative filenames and can therefore be a SECURITY risk
insecure RPATHs; these are relative RPATHs which are a SECURITY risk
the special $ORIGIN RPATHs are appearing after other RPATHs; this is just a minor issue but usually unwanted
the RPATH is empty; there is no reason for such RPATHs and they cause unneeded work while loading libraries
an RPATH references .. of an absolute path; this will break the functionality when the path before .. is a symlink

Opting out

A standard opt-out mechanism is provided, same as the other buildroot policy scripts, if the script provides incorrect results for your package. Simply add %define __brp_check_rpaths %{nil} on top of your SPEC. According to the guidelines, any package that disables a BRP script this way, MUST also note the reason in an accompanying comment.

Feedback

The change has been proposed a long time ago through FPC and the general consensus is that it needs to be done along with an overhaul of the Fedora documentation in regards to RPATH.

An email thread was also started on Fedora devel regarding this change.

There have been multiple requests in the past to enable that check, as well as various attempts to remove RPATH's from packages in the distro. 0 1 2 3

As for other distributions, Debian discourages the use of RPATH, openSUSE forbids it by running the check from rpmlint after every package build and Arch and Gentoo point out to possible insecure usage at their respective documentation pages.

Also there has been a relevant discussion in the upstream python community.

Benefit to Fedora

The main benefit of this change is avoiding bugs that might stem from hardcoded RPATH values which include but not limited to: loading of wrong or malicious code, wrong dependency resolution of object files etc. There will also be a performance increase by derived from avoiding unnecessary system calls required for handling RPATH's.

In addition the RPATH related packaging guidelines will be enforced by the build system.

Scope

Proposal owners: Merge the pull request to redhat-rpm-config to enable running the check-rpaths script after %install.

Other developers: After merging the changes to redhat-rpm-config the affected package maintainers that will see their packages' builds fail, will need to review their usage of RPATH and either remove it or workaround the issue. The packages currently failing to build due to RPATH issues, so far, are listed bellow:

Package list

audiofile

binutils

esc

ettercap

freeradius

fortune-mod

fcl

eb

conky-manager

condor

community-mysql

czmq

cfitsio

compat-guile18

glib2

gnokii

koffice-kivio

kicad

jq

komparator

k3guitune

laszip

levmar

hdf

gpick

kdepim3

gpgme

Io-language

kdegames3

gupnp-dlna

kdebase3

libcommuni

lutok

libburn

libminc

libisoburn

liboping

librfid

mingw-qt5-qtdeclarative

libkkc

openjade

libdxfrw

libosip2

libeXosip2

NLopt

libprelude

mingw-qt5-qt3d

mod_wsgi

libXcm

ncview

libdkimpp

mingw-qt5-qttools

mcpp

mingw-qt5-qtbase

mongo-c-driver

nightview

openscap

plotmm

pam_yubico

perl-SDL

pinentry

pam_mount

python2.7

rb_libtorrent

rrdtool

rarian

qwtpolar

qucs

scipy

tracker

SDL_image

sofia-sip

scap-workbench

woff2

xeus

yaz

stp

suitesparse

usnic-tools

sqlite2

vanessa_logger

xbsql

tracker-miners

WindowMaker

xmms

sylfilter

verbiste

zvbi

xdotool

texlive-base

zinnia

Release engineering: This change doesn't require coordination with rel-eng, as any issues will be caught during the regular mass rebuild of packages.

Policies and guidelines: TODO: The guidelines will be overhauled to take into account accepted usage or RPATH, clarification of the policy and ways to opt-out.

FPC ticket: https://pagure.io/packaging-committee/issue/886

Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

There will be no visible impact to non-packagers. Packagers will need to fix their packages if an broken RPATH entry was detected, as a broken RPATH will make the rpmbuild fail either in koji or locally.

How To Test

Mock build testing:

Initiate a Fedora rawhide mock chroot and install the modified redhat-rpm-config:

 mock -r fedora-rawhide-x86_64 --install https://download.copr.fedorainfracloud.org/results/cstratak/rpath/fedora-rawhide-x86_64/02160930-redhat-rpm-config/redhat-rpm-config-190-1.fc35.noarch.rpm

Build your package with the --no-clean option:

mock -r fedora-rawhide-x86_64 --no-clean <srpm>

Local testing: Building rpm's locally should already reveal the issue as the .rpmmacros file defines the RPATH check. Sample of a vanilla .rpmmacros file:

%_topdir %(echo $HOME)/rpmbuild
 
%__arch_install_post \
   [ "%{buildarch}" = "noarch" ] || QA_CHECK_RPATHS=1 ; \
   case "${QA_CHECK_RPATHS:-}" in [1yY]*) /usr/lib/rpm/check-rpaths ;; esac \
   /usr/lib/rpm/check-buildroot

rpmlint test: The binary-or-shlib-defines-rpath test from the BinariesCheck module will reveal RPATH usage.

 rpmlint -c BinariesCheck <binary rpm>

 e.g.: rpmlint -c BinariesCheck -v audiofile-0.3.6-27.fc34.x86_64.rpm 
       audiofile.x86_64: I: checking
       audiofile.x86_64: E: binary-or-shlib-defines-rpath /usr/bin/sfconvert ['/usr/lib64']
       audiofile.x86_64: E: binary-or-shlib-defines-rpath /usr/bin/sfinfo ['/usr/lib64']
       1 packages and 0 specfiles checked; 2 errors, 0 warnings.

User Experience

N/A (While this is a system wide change the impact is not visible to regular users, only to packagers as described above).

Dependencies

The change depends on modifying redhat-rpm-config.

Contingency Plan

Contingency mechanism: The change to redhat-rpm-config will be reverted
Contingency deadline: Beta freeze
Blocks release? No

Documentation

TODO:

The documentation of the packaging guidelines will be updated to reflect the changes that this change brings. Updated guidelines will be https://docs.fedoraproject.org/en-US/packaging-guidelines/#_beware_of_rpath

Release Notes

@@ Line 1: / Line 1: @@
 <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
-= Change Proposal Name =
+= Broken RPATH will fail rpmbuild =
-Enable RPATH detection during rpmbuild.
 == Summary ==
-Enable RPATH detection during rpmbuild. This will make the build fail in the case a hard-coded RPATH was detected within a binary or a shared library file.
+Enable broken RPATH detection [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_brp_buildroot_policy_scripts buildroot policy] script by default. This will make the RPM build fail once a broken RPATH was detected within a binary or a shared library file. An opt-out mechanism will be provided as well.
 == Owner ==
@@ Line 18: / Line 17: @@
 * FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
 -->
 == Current status ==
@@ Line 46: / Line 44: @@
 <!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
-The dynamic linker and loader (ld.so) is responsible for resolving runtime dependencies of executables and shared library files through a search hierarchy. However some packages (usually through their upstream buildsystems) contain a hard-coded path within their binaries or .so files, by using the -R or -rpath flag during compilation, which is referred to as RPATH. By utilizing RPATH, ELF files can point to directories to be included in the search path, on runtime, to resolve their dependencies.
+The dynamic linker and loader (ld.so) is responsible for resolving runtime dependencies of executables and shared library files through a search hierarchy. However some packages (usually through their upstream buildsystems) contain a hard-coded path within their binaries or .so files, by using the -R or -rpath flag during compilation, which is called an RPATH. By utilizing RPATH, ELF files can point to directories to be included in the search path, on runtime, to resolve their dependencies.
-While RPATH can be used for non-standard directories, such as a place containing private libraries of the project, when it points to a value already provided by the search path of ld.so it changes the hierarchy of the search, by placing the system defaults first.
+While RPATH can be used for non-standard directories, such as a place containing private libraries of the project, when it points to a value already provided by the search path of ld.so, it changes the hierarchy of the search by placing the system defaults first.
 (a) DT_RPATH -> (b) LD_LIBRARY_PATH -> (c) DT_RUNPATH -> (d) cache (/etc/ld.so.cache) -> (e) system defaults
-This could present a variety of issues, such as LD_LIBRARY_PATH overrides not working, incomplete dependency resolution, loading of wrong libraries etc. In general changing the default search hierarchy could lead to unforeseen bugs and issues (In a similar manner as as adding /usr/lib64 to LD_LIBRARY_PATH).
+This could present a variety of issues, such as LD_LIBRARY_PATH overrides not working, incomplete dependency resolution, loading of wrong libraries etc. In general, changing the default search hierarchy could lead to unforeseen bugs and issues - in a similar manner as adding /usr/lib64 to LD_LIBRARY_PATH.
 Another problem of a hardcoded RPATH is security. When an ELF object contains an RPATH pointed to a directory not managed by the system, where some malicious actor has write permissions to, it's relatively easy to execute arbitrary code.
-Performance can be affected as well, since probing explicitly e.g. /usr/lib64 through RPATH adds extra open/openat system calls to the process startup.
+Performance can be also affected, since probing explicitly e.g. /usr/lib64 through RPATH adds extra open/openat system calls to the process startup.
+In Fedora the use of such RPATH is [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_beware_of_rpath forbidden], but it was never enforced. This change intends to ratify that by executing `/usr/lib/rpm/check-rpaths` during rpmbuild, after %install, and fail the build if an RPATH entry was detected. This change will not affect RPATH's pointing to private libraries.
+=== Definition of a broken RPATH ===
+This change will use the [https://github.com/rpm-software-management/rpm/blob/master/scripts/check-rpaths-worker rpm script] for checking the broken RPATH's.
+The categories are:
+* standard RPATHs (e.g. `/usr/lib` or `/usr/lib64`); such RPATHs are a minor issue but are introducing redundant searchpaths without providing a benefit. They can also cause errors in multilib environments.
+*  invalid RPATHs; these are RPATHs which are neither absolute nor relative filenames and can therefore be a SECURITY risk
+*  insecure RPATHs; these are relative RPATHs which are a SECURITY risk
+*  the special `$ORIGIN` RPATHs are appearing after other RPATHs; this is just a minor issue but usually unwanted
+*  the RPATH is empty; there is no reason for such RPATHs and they cause unneeded work while loading libraries
+*  an RPATH references `..` of an absolute path; this will break the functionality when the path before `..` is a symlink
-In Fedora the use of RPATH is forbidden, but it was never enforced. This change intends to ratify that by running /usr/lib/rpm/check-rpaths during rpmbuild, after %install, and fail the build if an RPATH entry was detected.
+=== Opting out ===
+A standard opt-out mechanism is provided, same as the other [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_brp_buildroot_policy_scripts buildroot policy scripts], if the script provides incorrect results for your package. Simply add `%define __brp_check_rpaths %{nil}` on top of your SPEC. According to the guidelines, any package that disables a BRP script this way, MUST also note the reason in an accompanying comment.
 == Feedback ==
 <!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->
-The change [https://pagure.io/packaging-committee/issue/886 has been proposed] a long time ago through FPC and the general consensus is that it needs to be done, along with an overhaul of the Fedora documentation in regards to RPATH.
+The change [https://pagure.io/packaging-committee/issue/886 has been proposed] a long time ago through FPC and the general consensus is that it needs to be done along with an overhaul of the Fedora documentation in regards to RPATH.
 An [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/7ZKGVM4XJ7QFRFZXTSGUT4K2MPDVV2XY/#W7LXPX4SIB57DDXXI4PQNKCFSOQMOL4S email thread] was also started on Fedora devel regarding this change.
@@ Line 70: / Line 83: @@
 As for other distributions, Debian [https://wiki.debian.org/RpathIssue discourages] the use of RPATH, openSUSE [https://en.opensuse.org/openSUSE:Packaging_checks#Beware_of_Rpath forbids it] by running the check from rpmlint after every package build and Arch and Gentoo point out to possible insecure usage at their respective documentation pages.
+Also there has been a relevant [https://bugs.python.org/issue36659 discussion] in the upstream python community.
 == Benefit to Fedora ==
-Improved performance, security, compliance with the guidelines, reducing of RPATH related bugs.
+The main benefit of this change is avoiding bugs that might stem from hardcoded RPATH values which include but not limited to: loading of wrong or malicious code, wrong dependency resolution of object files etc. There will also be a performance increase by derived from avoiding unnecessary  system calls required for handling RPATH's.
+In addition the RPATH related packaging guidelines will be enforced by the build system.
 == Scope ==
-* Proposal owners: Merge the [https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/132 PR] to redhat-rpm-config to enable running the check-rpaths script after %install.
+* Proposal owners: Merge the [https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/132 pull request] to redhat-rpm-config to enable running the check-rpaths script after %install.
 <!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
@@ Line 267: / Line 283: @@
 |-
 |}
-* Release engineering: [https://pagure.io/releng/issues #Releng issue number] This change doesn't require coordination with rel-eng, as any issues will be caught during the regular mass rebuild of packages.<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Release engineering: This change doesn't require coordination with rel-eng, as any issues will be caught during the regular mass rebuild of packages.<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.
 The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
-* Policies and guidelines: The guidelines will be overhauled to take into account accepted usage or RPATH, clarification of the policy and ways to opt-out <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Policies and guidelines: TODO: The guidelines will be overhauled to take into account accepted usage or RPATH, clarification of the policy and ways to opt-out.
+FPC ticket: https://pagure.io/packaging-committee/issue/886<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. Please submit a pull request with the proposed changes before submitting your Change proposal. -->
@@ Line 278: / Line 295: @@
 == Upgrade/compatibility impact ==
 <!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
-There will be no visible impact to non-packagers. Packagers will need to fix their packages if an invalid RPATH entry was detected, as an invalid RPATH will make the rpmbuild fail either in koji or locally.
+There will be no visible impact to non-packagers. Packagers will need to fix their packages if an broken RPATH entry was detected, as a broken RPATH will make the rpmbuild fail either in koji or locally.
 == How To Test ==
-<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.
+* Mock build testing:
+Initiate a Fedora rawhide mock chroot and install the modified redhat-rpm-config:
+  mock -r fedora-rawhide-x86_64 --install https://download.copr.fedorainfracloud.org/results/cstratak/rpath/fedora-rawhide-x86_64/02160930-redhat-rpm-config/redhat-rpm-config-190-1.fc35.noarch.rpm
-Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
+Build your package with the --no-clean option:
+ mock -r fedora-rawhide-x86_64 --no-clean <srpm>
-A good "how to test" should answer these four questions:
+* Local testing: Building rpm's locally should already reveal the issue as the .rpmmacros file defines the RPATH check. Sample of a vanilla .rpmmacros file:
-. What special hardware / data / etc. is needed (if any)?
+ %_topdir %(echo $HOME)/rpmbuild
-. How do I prepare my system to test this change? What packages
-need to be installed, config files edited, etc.?
+ %__arch_install_post \
-. What specific actions do I perform to check that the change is
+    [ "%{buildarch}" = "noarch" ] || QA_CHECK_RPATHS=1 ; \
-working like it's supposed to?
+    case "${QA_CHECK_RPATHS:-}" in [1yY]*) /usr/lib/rpm/check-rpaths ;; esac \
-. What are the expected results of those actions?
+    /usr/lib/rpm/check-buildroot
--->
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* rpmlint test: The binary-or-shlib-defines-rpath test from the BinariesCheck module will reveal RPATH usage.
-* Mock testing:
+   rpmlint -c BinariesCheck <binary rpm>
-   Initiate a Fedora rawhide mock chroot and install the modified redhat-rpm-config: mock -r fedora-rawhide-x86_64 --install https://download.copr.fedorainfracloud.org/results/cstratak/rpath/fedora-rawhide-x86_64/02160930-redhat-rpm-config/redhat-rpm-config-190-1.fc35.noarch.rpm
-  Build your package with the --no-clean option: mock -r fedora-rawhide-x86_64 --no-clean <srpm>
-Local changes
+  e.g.: rpmlint -c BinariesCheck -v audiofile-0.3.6-27.fc34.x86_64.rpm
+        audiofile.x86_64: I: checking
+        audiofile.x86_64: E: binary-or-shlib-defines-rpath /usr/bin/sfconvert ['/usr/lib64']
+        audiofile.x86_64: E: binary-or-shlib-defines-rpath /usr/bin/sfinfo ['/usr/lib64']
+packages and 0 specfiles checked; 2 errors, 0 warnings.
 == User Experience ==
@@ Line 315: / Line 336: @@
 -->
-Packagers will have
+N/A (While this is a system wide change the impact is not visible to regular users, only to packagers as described above).
 == Dependencies ==
@@ Line 322: / Line 343: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-Merge the PR to redhat-rpm-config
+The change depends on modifying redhat-rpm-config.
 == Contingency Plan ==
 <!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
-* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Contingency mechanism: The change to redhat-rpm-config will be reverted
 <!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
-* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Contingency deadline: Beta freeze
 <!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
-* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Blocks release? No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-Revert the change
 == Documentation ==
@@ Line 339: / Line 358: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
+TODO:
+The documentation of the packaging guidelines will be updated to reflect the changes that this change brings. Updated guidelines will be https://docs.fedoraproject.org/en-US/packaging-guidelines/#_beware_of_rpath
 == Release Notes ==

Search

User:Cstratak/enable RPATH detection: Difference between revisions