Latest revision as of 16:31, 16 July 2020

Make the unversioned %{__python} macro error by default

Summary

The %{__python} RPM macro (currently defined to /usr/bin/python for backwards compatibility reasons) will be defined to raise an error when used. Any derived macros (%{python}, %{python_version}, %{python_sitleib} etc.) will propagate the error. Packagers can redefine the macro to any actual value to suppress the error. This is consistent with RHEL 8 behavior. Using /usr/bin/python in Fedora packages remains forbidden.

Owner

Name: Miro Hrončok
Email: <mhroncok@redhat.com>

Current status

Targeted release: Fedora 33
Last updated: 2020-07-16
FESCo issue: #2438
Tracker bug: #1857836
Release notes tracker: #534

Detailed Description

For years, the unversioned /usr/bin/python Python interpreter MUST not be used when building RPM packages in Fedora. However, for backwards compatibility reasons, the %{__python} macro was defined to /usr/bin/python. As a direct consequence, all derived macros:

%{python}
%{python_version}
%{python_version_nodots}
%{python_sitelib}
%{python_sitearch}
%py_shebang_fix
%py_build variants
%py_install variants

used /usr/bin/python as well, unless redefined to custom value different than /usr/bin/python. Some of the macros unfortunately evaluated to empty string when /usr/bin/python was not installed in the buildroot.

We wanted to define %{__python} to an error previously, but unfortunately, this was not yet possible due to backwards compatibility wrt automagic byte-compilation. Hence we have done:

Now, we can define the macro to an error by default. Packagers can still define it to any custom value.

We will define the macro as follows:

%__python %{error:attempt to use unversioned python, define %%__python to %{__python2} or %{__python3} explicitly}

This is technically consistent with RHEL 8.

We will also define %{python} to %{__python} (we will drop the current Lua logic that is designed to prevent %{python} usage when %{__python} is /usr/bin/python).

The default behavior will be an error:

$ rpm --eval '%__python'
error: attempt to use unversioned python, define %__python to /usr/bin/python2 or /usr/bin/python3 explicitly

$ rpm --eval '%python'
error: attempt to use unversioned python, define %__python to /usr/bin/python2 or /usr/bin/python3 explicitly

$ rpm --eval '%python_version'
error: attempt to use unversioned python, define %__python to /usr/bin/python2 or /usr/bin/python3 explicitly

$ rpm --eval '%python_sitelib'
error: attempt to use unversioned python, define %__python to /usr/bin/python2 or /usr/bin/python3 explicitly

As advised, when redefined, the macros continue to work as currently:

$ rpm --define '__python %__python3' --eval '%python'
/usr/bin/python3

$ rpm --define '__python %__python3' --eval '%python_version'
3.9

Despite the error message not actually promoting this, packagers can even explicitly define the macro to /usr/bin/python to mimic the previous behavior. However, this remains forbidden in Fedora.

$ rpm --define '__python /usr/bin/python' --eval '%python'
/usr/bin/python

$ rpm --define '__python /usr/bin/python' --eval '%python_sitelib'
/usr/lib/python3.9/site-packages

Feedback

Benefit to Fedora

More consistent behavior between RHEL and Fedora.
Avoids hard to debug mistakes when /usr/bin/python is not present and macros like %{python_sitelib} are used.
Doing the wrong thing is not the easiest default any more.

Scope

Proposal owners:
- Redefine %__python and %python

Other developers: nothing, AFAIK packages in Fedora already dropped this construct, however when not, packagers will need to define %__python in spec to make it work. We believe the error message is self-explanatory.

Release engineering: no impact

Policies and guidelines: Mostly already exist. The macro definition will need to be updated in the Python guidelines to match reality.

Trademark approval: not needed

Upgrade/compatibility impact

No user impact. Some spec files might start to fail to build with this change, but the error is self-explanatory.

How To Test

See examples in Detailed Description.

User Experience

No user impact. Some spec files might start to fail to build with this change, but the error is self-explanatory.

Dependencies

Changes/No_more_automagic_Python_bytecompilation_phase_3

Contingency Plan

Contingency mechanism: the change owners can revert the changes
Contingency deadline: beta freeze
Blocks release? No
Blocks product? No

Documentation

This page is the documentation. The updated macro list will also serve as documentation.

Release Notes

@@ Line 8: / Line 8: @@
 System Wide Changes are:
-* changes that does not fit Self Contained Changes category touching
+* changes that do not fit Self Contained Changes category touching
 * changes that require coordination within the distribution (for example mass rebuilds, release engineering or other teams effort etc.)
 * changing system defaults
@@ Line 43: / Line 43: @@
 == Current status ==
-[[Category:ChangeReadyForWrangler]]
+[[Category:ChangeAcceptedF33]]
 <!-- When your change proposal page is completed and ready for review and announcement -->
 <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
@@ Line 57: / Line 57: @@
 <!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page
 Bugzilla states meaning as usual:
-ASSIGNED -> accepted by FESCo with on going development
+ASSIGNED -> accepted by FESCo with ongoing development
 MODIFIED -> change is substantially done and testable
 ON_QA -> change is code completed and could be tested in the Beta release (optionally by QA)
 CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 -->
-* FESCo issue: <will be assigned by the Wrangler>
+* FESCo issue: [https://pagure.io/fesco/issue/2438 #2438]
-* Tracker bug: <will be assigned by the Wrangler>
+* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1857836 #1857836]
-* Release notes tracker: <will be assigned by the Wrangler>
+* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/534 #534]
 == Detailed Description ==
@@ Line 83: / Line 83: @@
 used <code>/usr/bin/python</code> as well, unless redefined to custom value different than <code>/usr/bin/python</code>. Some of the macros unfortunately evaluated to empty string when <code>/usr/bin/python</code> was not installed in the buildroot.
-We wanted to define <code>%{__python}</code> to an error previously, but unfortunately, this was not yet possible due to backwards compatibility wrt automagic byte compilation. Hence we have done:
+We wanted to define <code>%{__python}</code> to an error previously, but unfortunately, this was not yet possible due to backwards compatibility wrt automagic byte-compilation. Hence we have done:
 * [[Changes/No_more_automagic_Python_bytecompilation]]
@@ Line 89: / Line 89: @@
 * [[Changes/No_more_automagic_Python_bytecompilation_phase_3]]
-Now, we can define the macro to error by default. Packagers can still define it to any custom value.
+Now, we can define the macro to an error by default. Packagers can still define it to any custom value.
 We will define the macro as follows:
@@ Line 121: / Line 121: @@
 .9
-Despite the error message not actually promoting this, packagers can even explicitly define the macro to <code>/usr/bin/python</code> to mimic the previous behavior. However this remains forbidden in Fedora.
+Despite the error message not actually promoting this, packagers can even explicitly define the macro to <code>/usr/bin/python</code> to mimic the previous behavior. However, this remains forbidden in Fedora.
   $ rpm --define '__python /usr/bin/python' --eval '%python'
@@ Line 131: / Line 131: @@
 == Feedback ==
-<!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals, but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->
+<!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->
 == Benefit to Fedora ==
@@ Line 177: / Line 177: @@
 * Release engineering: no impact <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.
-The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
+The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
 * Policies and guidelines: Mostly already exist. The [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_macros macro definition] will need to be updated in the Python guidelines to match reality. <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->

Search

Changes/PythonMacroError: Difference between revisions