From Fedora Project Wiki
(draft)
 
No edit summary
Line 50: Line 50:
The is a refresh of Fedora 18 [[Features/DebugTypesSections]].
The is a refresh of Fedora 18 [[Features/DebugTypesSections]].


*.debug files contained in *-debuginfo.rpm are very big in general (debug/ has 82GB while all the other x86_64 Fedora distribution files are 75GB). There are various methods how to make the *-debuginfo.rpms smaller. Fedora 18 started using DWZ tool (from Features/DwarfCompressor) while [https://gcc.gnu.org/pipermail/gcc-patches/2008-August/246281.html Google implemented] a similar but different approach called -fdebug-types-section.
Debug info files *.debug contained in *-debuginfo.rpm are very big in general (debug/ has 82GB while all the other x86_64 Fedora distribution files are 75GB). There are various methods how to make the *-debuginfo.rpms smaller. Fedora 18 started using DWZ tool (from Features/DwarfCompressor) while [https://gcc.gnu.org/pipermail/gcc-patches/2008-August/246281.html Google implemented] a similar but different approach called -fdebug-types-section.


The problem of existing Fedora DWZ saving is that almost nobody uses it (only Fedora/CentOS/RHEL and SuSE OSes) and so its support is missing in tools like [https://lldb.llvm.org/ LLDB] and [[llvm-dwarfdumphttps://llvm.org/docs/CommandGuide/llvm-dwarfdump.html|llvm-dwarfdump]]. Implementing support for DWZ is about 8x times more complicated (LoC count) than -fdebug-types-section.
Almost nobody uses existing Fedora DWZ (only Fedora/CentOS/RHEL and SuSE OSes) and so its support is missing in tools like [https://lldb.llvm.org/ LLDB] and [[llvm-dwarfdumphttps://llvm.org/docs/CommandGuide/llvm-dwarfdump.html|llvm-dwarfdump]].


* DWZ advantage: On the whole Fedora distro it saves 3.3% (5GB of the 157GB distribution size).
* DWZ advantage: On the whole Fedora distro it saves 3.3% (5GB of the 157GB distribution size)
* Bulleted list item
  * If the 3.3% size increase is a concern I can implement a different optimization ([https://whova.com/embedded/session/llvm_202010/1193947/ talk (2)]) as GCC post-proccessing phase which requires no support in consumers.
* DWZ disadvantage: Compilation (linking) requires 2x as big disk space (as DWZ is processing files after linker and DWZ is incompatible with -fdebug-types-section)
* DWZ disadvantage: Compilation (linking) is slower
* DWZ disadvantage: DWZ requires 8x times more complicated (LoC count) support in consumers than -fdebug-types-section.
* DWZ disadvantage: DWZ has currently less support across consumers (LLDB, llvm-dwarfdump, binutils readelf)
* DWZ disadvantage: DWZ cannot update LLVM .debug_names index which can be generated only by clang (it cannot be regenerated later for DWZ-compressed file)
* DWZ disadvantage: DWZ DWARF-5 support is a work-in-progress and there is no plan to support DWARF-5 produced by clang


== Feedback ==
== Feedback ==
Line 94: Line 100:
* Proposal owners:
* Proposal owners:
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
It affects all packages generating *-debuginfo.rpm.


* Other developers: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers:
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
Report any possible debuginfo incompatibility (unexpected).


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Line 102: Line 110:
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->


* Policies and guidelines: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines:
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
All the needed changes should be done in [https://src.fedoraproject.org/rpms/redhat-rpm-config redhat-rpm-config]. The [https://src.fedoraproject.org/rpms/dwz dwz package] can be then retired.


* Trademark approval: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
Line 110: Line 119:
* Alignment with Objectives:  
* Alignment with Objectives:  
<!-- Does your proposal align with the current Fedora Objectives: https://docs.fedoraproject.org/en-US/project/objectives/ ? It's okay if it doesn't, but it's something to consider -->
<!-- Does your proposal align with the current Fedora Objectives: https://docs.fedoraproject.org/en-US/project/objectives/ ? It's okay if it doesn't, but it's something to consider -->
The size differences are only for *-debuginfo.rpm which is outside of scope of the listed objectives.


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
Line 115: Line 125:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
As *-debuginfo.rpm must exactly match NVRA of its binary package the compatibility is not relevant. Existing tools supporting DWZ will still support the DWZ file format in packages which have not been rebuilt.


== How To Test ==
== How To Test ==
Line 132: Line 142:
-->
-->


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
It is not a big risk, -fdebug-types-section is more widespread and more simple than DWZ. Still I should run:
N/A (not a System Wide Change)
* [https://src.fedoraproject.org/rpms/libabigail libabigail] DWARF comparisons
* testsuites of tools like: clang, lldb, gcc, binutils, gdb, elfutils, rpm


== User Experience ==
== User Experience ==
Line 146: Line 157:
  - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
  - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
-->
-->
* Better compatibility with existing debugging and tracing tools.
* Less resource-intensive rebuilds of C++ packages (both in disk space, memory requirements and compilation time).


== Dependencies ==
== Dependencies ==
<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this change depends?  In other words, completion of another change owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel change)? -->
<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this change depends?  In other words, completion of another change owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel change)? -->
 
none
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)


== Contingency Plan ==
== Contingency Plan ==


<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Contingency mechanism: Nothing needed, Fedora can continue using DWZ, just some debugging/tracing tools will continue to be incompatible.
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Contingency deadline: beta freeze
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks release? No
* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
* Blocks product? N/A


== Documentation ==
== Documentation ==
<!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
<!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* [http://www.dwarfstd.org/doc/DWARF5.pdf DWARF-5] E.2 Using Type Units
N/A (not a System Wide Change)
* [https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#index-fdebug-types-section GCC -fdebug-types-section]


== Release Notes ==
== Release Notes ==

Revision as of 14:26, 23 September 2020


Debug Info Standardization

Summary

Fedora 18 implemented Features/DwarfCompressor. As the format did not get widespread and the tool is not much maintained it became burden to make existing debugging tools compatible with Fedora.

Owner

Current status

  • Targeted release: Fedora 34
  • Last updated: 2020-09-23
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

The is a refresh of Fedora 18 Features/DebugTypesSections.

Debug info files *.debug contained in *-debuginfo.rpm are very big in general (debug/ has 82GB while all the other x86_64 Fedora distribution files are 75GB). There are various methods how to make the *-debuginfo.rpms smaller. Fedora 18 started using DWZ tool (from Features/DwarfCompressor) while Google implemented a similar but different approach called -fdebug-types-section.

Almost nobody uses existing Fedora DWZ (only Fedora/CentOS/RHEL and SuSE OSes) and so its support is missing in tools like LLDB and llvm-dwarfdump.

  • DWZ advantage: On the whole Fedora distro it saves 3.3% (5GB of the 157GB distribution size)
 * If the 3.3% size increase is a concern I can implement a different optimization (talk (2)) as GCC post-proccessing phase which requires no support in consumers.
  • DWZ disadvantage: Compilation (linking) requires 2x as big disk space (as DWZ is processing files after linker and DWZ is incompatible with -fdebug-types-section)
  • DWZ disadvantage: Compilation (linking) is slower
  • DWZ disadvantage: DWZ requires 8x times more complicated (LoC count) support in consumers than -fdebug-types-section.
  • DWZ disadvantage: DWZ has currently less support across consumers (LLDB, llvm-dwarfdump, binutils readelf)
  • DWZ disadvantage: DWZ cannot update LLVM .debug_names index which can be generated only by clang (it cannot be regenerated later for DWZ-compressed file)
  • DWZ disadvantage: DWZ DWARF-5 support is a work-in-progress and there is no plan to support DWARF-5 produced by clang

Feedback

Benefit to Fedora

Scope

  • Proposal owners:

It affects all packages generating *-debuginfo.rpm.

  • Other developers:

Report any possible debuginfo incompatibility (unexpected).

  • Policies and guidelines:

All the needed changes should be done in redhat-rpm-config. The dwz package can be then retired.

  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Objectives:

The size differences are only for *-debuginfo.rpm which is outside of scope of the listed objectives.

Upgrade/compatibility impact

As *-debuginfo.rpm must exactly match NVRA of its binary package the compatibility is not relevant. Existing tools supporting DWZ will still support the DWZ file format in packages which have not been rebuilt.

How To Test

It is not a big risk, -fdebug-types-section is more widespread and more simple than DWZ. Still I should run:

* libabigail DWARF comparisons
* testsuites of tools like: clang, lldb, gcc, binutils, gdb, elfutils, rpm

User Experience

* Better compatibility with existing debugging and tracing tools.
* Less resource-intensive rebuilds of C++ packages (both in disk space, memory requirements and compilation time).

Dependencies

none

Contingency Plan

  • Contingency mechanism: Nothing needed, Fedora can continue using DWZ, just some debugging/tracing tools will continue to be incompatible.
  • Contingency deadline: beta freeze
  • Blocks release? No
  • Blocks product? N/A

Documentation

* DWARF-5 E.2 Using Type Units
* GCC -fdebug-types-section

Release Notes