From Fedora Project Wiki
No edit summary
Line 49: Line 49:


== Detailed Description ==
== Detailed Description ==
After https://fedoraproject.org/wiki/Changes/Python_Extension_Flags we uncoupled some distro specific compilation and linker flags from distutils, meaning users who build their own Python C extensions wouldn't get some Fedora specific flags such as the annobin ones.
After implementing https://fedoraproject.org/wiki/Changes/Python_Extension_Flags we uncoupled some distro specific compilation and linker flags propagated to C extensions.


However with an ever increasing set of compiler flags being applied on top as compilers and security standards evolve (e.g. -D_FORTIFY_SOURCE=3) it becomes a heavier burden on the python maintainers to vet each change that might leak into distutils which then applied on user-built C extensions. Instead of removing only some flags we will remove all of them that don't preserve binary compatibility with the main interpreter.
However with an ever increasing set of compiler flags being added and applied distro-wide, as compilers and security standards evolve (e.g. -D_FORTIFY_SOURCE=3) it becomes an increasingly complex job to vet each flag that might leak into user-built Python C extensions through the Python interpreter. Instead of removing only some flags and letting the rest follow through, we will be taking a more proactive approach by removing all the compiler and linker flags, except the ones that are required to maintain the binary compatibility with the Python interpreter the extensions were built against which is <code>-fexceptions</code>. We will also preserve the ones that Python hardcodes itself through the Makefile.
 
Similarly, when a user builds their own C programs, no compiler flags are applied by default and the user is free to making their own decision. Bringing the compilation of Python C extensions closer to that experience is the next logical step.
 
Currently a user-built Python C extension will be built with:
 
CFLAGS:
<code>
-Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS  -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection    -D_GNU_SOURCE -fPIC -fwrapv -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS  -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection    -D_GNU_SOURCE -fPIC -fwrapv  -O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS  -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection    -D_GNU_SOURCE -fPIC -fwrapv
</code>
 
LDFLAGS:
<code>
'-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now    -Wl,--build-id=sha1  -g  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now    -Wl,--build-id=sha1  -g'
</code>
 
After this change:
 
CFLAGS:
<code>
-Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -fexceptions -fexceptions -fexceptions
</code>
 
LDFLAGS:
None


Similarly, when a user builds their own C programs, no compiler flags are applied by default and the user is free to decide. Syncing building Python C extensions to that experience is the next logical step.
== Feedback ==
== Feedback ==
<!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->
<!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->

Revision as of 22:42, 15 June 2023

Further reduce Fedora-specific build flags in non-RPM Python extensions

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Continuing the work started with https://fedoraproject.org/wiki/Changes/Python_Extension_Flags, this change is about further reducing the build and linker flags (CFLAGS, CXXFLAGS and LDFLAGS) saved internally in the Python interpreter for use by distutils and other build systems. Compiling non-RPM Python extension modules will carry only the compiler flags required for binary compatibility with the interpreter they were built against and not Fedora specific ones.

Practically that means the only Fedora derived flag will be -fexceptions and Python will apply its own upstream hardcoded ones, making the final flag set for a non-RPM compiled Python extension as follows:

  • -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -fexceptions

The current main Python interpreter on Fedora 39 will be modified (Python 3.12) and Python 3.6-3.11 will follow.

Owner


Current status

  • Targeted release: Fedora Linux 39
  • Last updated: 2023-06-15
  • [<will be assigned by the Wrangler> devel thread]
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

After implementing https://fedoraproject.org/wiki/Changes/Python_Extension_Flags we uncoupled some distro specific compilation and linker flags propagated to C extensions.

However with an ever increasing set of compiler flags being added and applied distro-wide, as compilers and security standards evolve (e.g. -D_FORTIFY_SOURCE=3) it becomes an increasingly complex job to vet each flag that might leak into user-built Python C extensions through the Python interpreter. Instead of removing only some flags and letting the rest follow through, we will be taking a more proactive approach by removing all the compiler and linker flags, except the ones that are required to maintain the binary compatibility with the Python interpreter the extensions were built against which is -fexceptions. We will also preserve the ones that Python hardcodes itself through the Makefile.

Similarly, when a user builds their own C programs, no compiler flags are applied by default and the user is free to making their own decision. Bringing the compilation of Python C extensions closer to that experience is the next logical step.

Currently a user-built Python C extension will be built with:

CFLAGS: -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv

LDFLAGS: '-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g'

After this change:

CFLAGS: -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -fexceptions -fexceptions -fexceptions

LDFLAGS: None

Feedback

Benefit to Fedora

Scope

  • Proposal owners:
  • Other developers:
  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Community Initiatives:

Upgrade/compatibility impact

How To Test

User Experience

Dependencies

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? N/A (not a System Wide Change), Yes/No


Documentation

N/A (not a System Wide Change)

Release Notes

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/Python_Extension_Flags_Reduction&oldid=680263"