From Fedora Project Wiki

Revision as of 15:06, 10 September 2018 by Churchyard (talk | contribs)



Make ambiguous python shebangs error

Summary

The /usr/lib/rpm/redhat/brp-mangle-shebangs buildroot policy script will be changed to make the build fail when it sees an ambiguous python shebang, such as #!/usr/bin/python or #!/usr/bin/env python. (The script has been warning in these cases for 2 Fedora releases already, saying This will become an ERROR.)

Owner

  • Name: Miro Hrončok
  • Email: <mhroncok@redhat.com>
  • Release notes owner:

Current status

Detailed Description

The buildroot policy script in /usr/lib/rpm/redhat/brp-mangle-shebangs currently changes all python shebangs to python2 with a message like:

*** WARNING: mangling shebang in /usr/bin/taskotron_result from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!

We will change it to:

*** ERROR: ambiguous python shebang in /usr/bin/taskotron_result: #!/usr/bin/python. Change it to python3 (or python2) explicitly.

The script will exit with nonzero exit code, rendering the build failed.

The warning and a promise of the error was there for 2 releases (28 and 29). Taskotron check was also present.

There are standard mechanics to avoid this buildroot policy script or to block certain files form it. Those remain intact by this change. For details see Shebang lines and BuildRoot Policy Scripts sections of the packaging guidelines.

Using pathfix.py to fix shebangs

Packagers are encouraged to use pathfix.py to fix shebangs. Here is an example. First, buildrequire the tool:

BuildRequires: /usr/bin/pathfix.py

Note: The examples bellow use some python macros (%{__pythonX} and %{pyX_shbang_opts}). BuildRequire python3-devel (and/or python2-devel) to get those.

Fix the shebangs ideally in prep:

%prep
...setup...

# Fix all Python shebangs recursively in .
# -p preserves timestamps
# -n prevents creating ~backup files
# -i specifies the interpreter for the shebang
# Need to list files that do not match ^[a-zA-Z0-9_]+\.py$ explicitly!
pathfix.py -pni "%{__python3} %{py3_shbang_opts}" . whatnot/file-with-dashes.py else/no_py_extension

Sometimes shebnaged files are created "magically" during install, so the fix needs to be applied after that:

%install
...install stuff...

# Some files got ambiguous python shebangs, we fix them after everything else is done
pathfix.py -pni "%{__python2} %{py2_shbang_opts}" %{buildroot}%{python2_sitearch}
pathfix.py -pni "%{__python3} %{py3_shbang_opts}" %{buildroot}%{python3_sitearch} %{buildroot}%{_bindir}/*

Benefit to Fedora

Packagers will be notified by build error if they accidentally have python2 shebangs (and python2 dependency) they didn't anticipate. It's up to them to decide what to do with such files, no automation can know.

Scope

  • Other developers: fix their packages if they fail because of this
  • Release engineering: #7679 (a check of an impact with Release Engineering is needed)
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

Have an RPM package that tries to ship files with ambiguous python shebang. Observe the warning on Fedora 29 and the error on Fedora 30.

User Experience

Users should not notice this, expect there might be less unneeded python2 dependencies created by accident.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? N/A (not a System Wide Change)
  • Blocks product? product

Documentation

The Python Packaging Guidelines will be the documentation.

Release Notes

TBD