Changes/ZlibNGTransition: Difference between revisions

Latest revision as of 20:12, 28 February 2024

Transitioning to Zlib-ng as compatible replacement for Zlib

Summary

Replace Zlib with Zlib-ng. This change is strongly tied to Changes/MinizipNGTransition.

Owner

Name: Tulio Magno Quites Machado Filho, Lukas Javorsky, Ali Erdinc Koroglu, Florian Weimer, Davide Cavalca, Neal Gompa
Email: tuliom@redhat.com, ljavorsk@redhat.com, ali.erdinc.koroglu@intel.com, fweimer@redhat.com, davide@cavalca.name, ngompa13@gmail.com

Current status

Targeted release: Fedora Linux 40
Last updated: 2024-02-28
Announced
Discussion Thread
FESCo issue: #3109
Tracker bug: #2252767
Release notes tracker: #1083

Detailed Description

Zlib has long served as a reliable compression library, but its performance on modern CPU architectures does not fully utilize their capabilities. The transition to Zlib-ng aims to enhance the compression efficiency and performance for Fedora while ensuring compatibility with existing packages and libraries.

Feedback

Benefit to Fedora

Zlib-ng supports hardware acceleration when available
- Adler32 implementation using SSSE3, AVX2, AVX512, AVX512-VNNI, Neon, VMX & VSX
- CRC32-B implementation using PCLMULQDQ, VPCLMULQDQ, ACLE, & IBM Z
- Hash table implementation using CRC32-C intrinsics on x86 and ARM
- Slide hash implementations using SSE2, AVX2, ARMv6, Neon, VMX & VSX
- Compare256 implementations using SSE2, AVX2, Neon, POWER9 & RVV
- Inflate chunk copying using SSE2, SSSE3, AVX, Neon & VSX
- Support for hardware-accelerated deflate using IBM Z DFLTCC
Includes Intel and Cloudflare optimizations
Uses code sanitizers, fuzzing, and code coverage

Scope

Proposal owners:
- Zlib-ng will be rebuilt with zlib-compatible API, obsoleting zlib-1.2
  - Zlib-ng compat mode will be distributed in a new package called zlib-ng-compat, with its respective zlib-ng-compat-devel.
  - Initially, zlib-ng-compat will provide zlib = 1.2.13 and zlib-ng-compat-devel will provide zlib-devel = 1.2.13.
  - These versions will be updated following the support added to zlib-ng upstream.
- Considering the new packages promise to be ABI-compatible with zlib, the packages depending on zlib won't be rebuilt.
  - This may help to catch ABI-compatibility issues soon.
Other developers: Help with build failures may be requested.
Release engineering: No action required
Policies and guidelines: N/A
Trademark approval: N/A
Alignment with Objectives: N/A

Upgrade/compatibility impact

Zlib-ng has a zlib-compat mode that aims to preserve both API and ABI compatibility with the original Zlib. However certain application tests may rely on hardcoded checksums for the compressed output. These tests could fail due to various changes, even if the compressed output remains valid and decompresses correctly.

How To Test

Update the system, verify that it does not break other packages.

User Experience

This change will increase compression efficiency and performance

Dependencies

Zlib is a dependency for over 100 packages. Most of them are expected not to fail or break.

Contingency Plan

Contingency mechanism: Revert the change to Zlib and rebuild dependent packages.
Contingency deadline: Beta freeze
Blocks release? No

Documentation

Release Notes

@@ Line 1: / Line 1: @@
-= Transitioning to Zlib-ng as compatible replacement for Zlib =
+== Transitioning to Zlib-ng as compatible replacement for Zlib ==
-{{Change_Proposal_Banner}}
 == Summary ==
-Replace Zlib with Zlib-ng and minizip with minizip-ng.
+Replace Zlib with Zlib-ng. This change is strongly tied to [[Changes/MinizipNGTransition]].
 == Owner ==
@@ Line 11: / Line 10: @@
 == Current status ==
-[[Category:ChangePageIncomplete]]
+[[Category:ChangeAcceptedF40]]
 <!-- When your change proposal page is completed and ready for review and announcement -->
 <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
@@ Line 28: / Line 27: @@
 ON_QA -> change is fully code complete
 -->
-* FESCo issue: <will be assigned by the Wrangler>
+* [https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/75A7Y4X3RLY357T246IOQQ2UNSSVS5IW/ Announced]
-* Tracker bug: <will be assigned by the Wrangler>
+* [https://discussion.fedoraproject.org/t/f40-change-proposal-transitioning-to-zlib-ng-as-a-compatible-replacement-for-zlib-system-wide/95807 Discussion Thread]
-* Release notes tracker: <will be assigned by the Wrangler>
+* FESCo issue: [https://pagure.io/fesco/issue/3109 #3109]
+* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=2252767 #2252767]
+* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/1083 #1083]
 == Detailed Description ==
 Zlib has long served as a reliable compression library, but its performance on modern CPU architectures does not fully utilize their capabilities. The transition to Zlib-ng aims to enhance the compression efficiency and performance for Fedora while ensuring compatibility with existing packages and libraries.
-Meanwhile, minizip-ng received new features and bug fixes.
 == Feedback ==
@@ Line 51: / Line 50: @@
 * Includes Intel and Cloudflare optimizations
 * Uses code sanitizers, fuzzing, and code coverage
-* minizip-ng has the following features
-** Zlib, BZIP2, LZMA, XZ, and ZSTD compression methods.
-** Password protection through Traditional PKWARE and WinZIP AES encryption.
-** Buffered streaming for improved I/O performance.
-** NTFS timestamp support for UTC last modified, last accessed, and creation dates.
-** Disk split support for splitting zip archives into multiple files.
-** Preservation of file attributes across file systems.
-** Follow and store symbolic links.
-** Unicode filename support through UTF-8 encoding.
-** Legacy character encoding support CP437, CP932, CP936, CP950.
-** Streaming interface for easy implementation of additional platforms.
 == Scope ==
@@ Line 68: / Line 55: @@
 ** Zlib-ng will be rebuilt with zlib-compatible API, obsoleting zlib-1.2
 *** Zlib-ng compat mode will be distributed in a new package called zlib-ng-compat, with its respective zlib-ng-compat-devel.
-*** Initially, zlib-ng-compat will provide zlib = 1.2.13 and zlib-ng-compat-devel will provide zlib = 1.2.13.
+*** Initially, zlib-ng-compat will provide zlib = 1.2.13 and zlib-ng-compat-devel will provide zlib-devel = 1.2.13.
 *** These versions will be updated following the support added to zlib-ng upstream.
 ** Considering the new packages promise to be ABI-compatible with zlib, the packages depending on zlib won't be rebuilt.
 *** This may help to catch ABI-compatibility issues soon.
 * Other developers: Help with build failures may be requested.
-* Release engineering: TODO
+* Release engineering: No action required
 * Policies and guidelines: N/A
 * Trademark approval: N/A
@@ Line 93: / Line 80: @@
 * Contingency mechanism: Revert the change to Zlib and rebuild dependent packages.
 * Contingency deadline: Beta freeze
-* Blocks release? TODO
+* Blocks release? No
 == Documentation ==
-TODO
 == Release Notes ==

Search