Changes/Making sudo pip safe

Making sudo pip Safe (Again)

Summary

At the present time, running sudo pip3 in Fedora is not safe. Pip shares its installation directory with dnf, can remove dnf-managed files and generally break the Python 3 interpreter. We propose a series of measures that will make it safe to use.

Owner

• Name: Michal Cyprian
• Name: Petr Viktorin
• Name: Tomas Orsava
• Name: Miro Hroncok
• Email: python-maint@redhat.com
• Release notes owner:

Current status

• Targeted release: Fedora 26
• Last updated: 2016-12-14
• Tracker bug: <will be assigned by the Wrangler>

Detailed Description

The danger of using sudo pip3 stems from the fact that both Python dnf packages and sudo pip3 install modules to the same location, namely /usr/lib/pythonX.Y/site-packages.

We aim to move the working directory for sudo pip3 to a more appropriate location: /usr/local/lib/pythonX.Y/site-packages, and modify the Python 3 interpreter in Fedora to scan both above mentioned locations when importing modules. In addition, system-python—a stripped down version of Python 3 for use by system tools—will not read the sudo pip3 install location, making it more secure by being less susceptible to interference by user-downloaded modules.

From the technical standpoint, this will be accomplished by changing the sys.prefix setting in the /usr/bin/python3 executable from /usr/ to /usr/local. pip3 will thereafter use this prefix when determining where to install modules. In addition, the original path /usr/lib/pythonX.Y/site-packages will be added to the sys.path variable (so that modules at that location are still processed when importing), because this path will not be automatically scanned anymore as it no longer lies inside the sys.prefix path. These settings, however, will not be modified for the system-python binary, and the %{__python3} macro will be changed from /usr/bin/python3 to /usr/libexec/system-python. Therefore, Python dnf packages will continue to be built with the correct installation path for system modules.

Note that using sudo pip3 is not strictly necessary, as using pip3 install --user would satisfy the vast majority of use cases. Nevertheless, sudo pip is far too prevalent an instruction in various guides and installation notes throughout the Internet that there is little hope of changing users' behaviour in this regard.

Benefit to Fedora

Fedora users will benefit from the increased stability of the ecosystem. No longer will they wonder why their Python interpreter or system modules don't work after using sudo pip [1][2][3][4].

Additionally, Fedora will increase it's compliance with the Filesystem Hierarchy Standard as user-installed host-specific Python modules will now be correctly located under /usr/local.

Scope

• Proposal owners:
  • Modify the Python 3 executable as described above.
  • Modify the %{__python3} macro so that it points to /usr/libexec/system-python

• Other developers: Spec files that use pip3 install without the use of a macro will need to be modified accordingly. Only 3 like packages were identified (python-flit, python-entrypoints, python-setuptools).

• Release engineering: A rebuild of all Python packages will be necessary.
  • List of deliverables: All Fedora deliverables will be affected in a minor way that does not jeopardize their delivery.

• Policies and guidelines: The definition of the %{__python3} macro will be updated as mentioned above.

• Trademark approval: Not needed for this Change

Upgrade/compatibility impact

Upgraded systems will continue to work as expected. Modules previously installed using sudo pip3 will remain in the location now reseved for system modules, and will need to be manually deleted if removal is required. However, they won't pose a threat to the stability of the system.

How To Test

Test dnf-installed modules

1. Install any Python 3 module using dnf.

2. Check if a corresponding file or directory was created at /usr/lib/pythonX.Y/site-packages/.

3. Run python3 and try to import said module.

4. Run /usr/libexec/system-python and try to import said module.

Test sudo pip3–installed modules

1. Install any Python 3 module using sudo pip3

2. Check if a corresponding file or directory was created at /usr/local/lib/pythonX.Y/site-packages/.

3. Run python3 and try to import said module.

4. Run /usr/libexec/system-python and try to import said module—the import should fail.

User Experience

Running sudo pip3 won't randomly result in a broken Python 3 on the user's system.

Dependencies

N/A

Contingency Plan

• Contingency mechanism: Do not push changes to python3 and Python macros into rawhide.
• Contingency deadline: Alpha freeze
• Blocks release? No

Documentation

Discussions

0. python-devel Fedora mailing list

1. pypa-dev mailing list

Release Notes

The location where sudo pip3 installs modules has been changed to /usr/local/lib/pythonX.Y/site-packages, and sudo pip3 is henceforth safe to use. No other changes in user experience are expected.