From Fedora Project Wiki



Move /usr/bin/python into a separate package

Summary

Reflecting the recent changes of PEP 394 -- The "python" Command on Unix-Like Systems, we are moving /usr/bin/python from the python2 package into a separate package called python-unversioned-command. python2 will recommend this package.

This means Fedora users will get it automatically with python2, but they might opt-out and remove it. In Fedora's build system, only packages explicitly buildrequiring /usr/bin/python will get it.

This change obsoletes "Avoid /usr/bin/python in RPM build" change.


Owner

  • Release notes owner:

Current status


Detailed Description

Motivation

The meaning of the python command is ambiguous: it might mean either Python 2 or Python 3, depending on context. The upstream recommendation (PEP 394), which we try to follow in Fedora, is that users -- not distros, and not sysadmins -- should be in control of the python command.

Specifically, this means distro-packaged software should use python2 or python3 explicitly. Fedora's Python packaging guidelines have suggested this since February 2015, and demand it since August 2017. However, enforcing this guideline (which we will need to actually allow users to safely change their python command) is problematic. "Avoid /usr/bin/python in RPM_Build" change didn't work: most of the packagers didn't change anything and too many packages still use the python command.

Instead of developing custom hacks, we are now utilizing a more systematic solution: If your package still needs /usr/bin/python to build, explicitly BuildRequire it. If your package needs /usr/bin/python to run, explicitly Require it. (In both cases, try to use /usr/bin/python2 or /usr/bin/python3 instead if possible.)

We are also expecting some buildsystems (autools, cmake, etc.) to automatically use /usr/bin/python2 if /usr/bin/python is unavailable, so the problem might go away more naturally.

PEP 394 and how it is mapped to this change

Upstream PEP 394 -- The "python" Command on Unix-Like Systems describes how the python command should behave. In Fedora, that's /usr/bin/python. The PEP was recently updated to reflect upstream's evolving views on the situation. Notable new information from the PEP is: 

In controlled environments aimed at expert users, where being explicit is valued over user experience (for example, in test environments and package build systems), distributions may choose to not provide the python command even if python2 is available. (All software in such a controlled environment must use python3 or python2 rather than python, which means scripts that deliberately use python need to be modified for such environments.)

We consider Fedora's build machinery a controlled environments aimed at expert users.

What's changing

"Avoid /usr/bin/python in RPM_Build" change is reverted. Packages that follow its Quick Opt-Out section (i.e. set PYTHON_DISALLOW_AMBIGUOUS_VERSION) will be fixed by the owners of this change.

/usr/bin/python remains a symbolic link to /usr/bin/python2. However, it is moved to a new python-unversioned-command package (technically a subpackage of python2). python2 package will only recommend /usr/bin/python.

Packages that need /usr/bin/python to build will need to BuildRequire it. Packages that need /usr/bin/python to be used by users will need to Require it. In both cases, the packager should avoid the need and only fallback to (Build)Requiring /usr/bin/python as a temporary workaround.

The new package will virtually provide python, ensuring that dnf install python will make the python command available.

Effect on automatic bytecompilation

"No more automagic Python bytecompilation" change is done, packages that byte-compile files outside of Python directories should switch to the new behavior described in that change, and should not be impacted by this change. However, if that change is delayed or reverted, packagers that rely on the old behavior when byte compiling files will need to set %__python to python2 or python3 explicitly.

Effect on %__python and other ambiguous RPM macros

Using ambiguous Python macros (%{__python}, %{python_sitelib}...) is forbidden and your package will fail to build if you still use those without redefining %__python. Either switch to explicitly versioned macros (%{__python2}, %{python2_sitelib}, %{__python3}...) or set __python to an explicit Python version.

Don't (Build)Require python-unversioned-command, but /usr/bin/python

If this change ever gets obsoleted, or for older Fedoras support, we ask the maintainers who fallback to (Build)Requiring /usr/bin/python, not to use the package name for the dependency, but the executable path instead.

Benefit to Fedora

This change brings us one step closer to a seamless transition to Python 3, while following upstream recommendations.

This change allows experienced users to remove /usr/bin/python in a supported way.

Scope

  • Proposal owners:
    • Split the packages as described in Detailed Description.
    • Fix packages that use PYTHON_DISALLOW_AMBIGUOUS_VERSION to BuildRequire /usr/bin/python instead.
  • Other developers:
    • Maintainers of packages that use /usr/bin/python need to switch to using /usr/bin/python3 or /usr/bin/python2 explicitly (with help from Proposal owners if needed).
      • While doing that, consider switching your package to Python 3 only, if the Python 2 bits are unused in Fedora. (This is not necessarily required for this change, however it will make your packaging job easier.)
      • If that can't be done in a timely manner, fallback to (Build)Requiring /usr/bin/python as a temporary workaround.
  • Release engineering: #7511 (a check of an impact with Release Engineering is needed)
  • Policies and guidelines:
    • Already existing: "packages in Fedora ... MUST call the proper executable for the needed python major version directly, either /usr/bin/python2 or /usr/bin/python3 as appropriate" from Packaging:Python#Multiple_Python_Runtimes.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

All packages that rely on /usr/bin/python need to change to /usr/bin/python2, or add explicit (Build)Requires.

Advanced users that don't install recommended packages by default will either have to install /usr/bin/python manually or live without it. Generally, the users should not be affected by this change.

How To Test

TBD

User Experience

Advanced users might uninstall the /usr/bin/python link if they want to. Others don't need to deal with this change.

Dependencies

We expect packages to FTBFS because of this. Packagers need to fix them (with help from Change owners if requested). If they don't fix them, the usual FTBFS policy applies.

This change will be done after the switch to Python 3.7 to avoid work on one change from interfering with work on the other.

Contingency Plan

  • Contingency mechanism: If this proves problematic, we will make python2 require /usr/bin/python (instead of just recommending it).
  • Contingency deadline: beta freeze
  • Blocks release? No
  • Blocks product? No

Documentation

TBD

Release Notes

TBD

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/Move_usr_bin_python_into_separate_package&oldid=521906"