(Created page with ' = XZ (LZMA) Payloads in RPM = <!-- The name of your feature --> == Summary == We intend to switch RPM to use XZ (the new LZMA format) by default for F12. == Owner == * Name: [[...') |
No edit summary |
||
| (32 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
= XZ (LZMA) Payloads in RPM = | = XZ (LZMA) Payloads in RPM = | ||
<!-- The name of your feature --> | <!-- The name of your feature --> | ||
== Summary == | == Summary == | ||
We intend to switch RPM to use XZ (the new LZMA format) | We intend to switch RPM to use XZ (the new LZMA format) as default payload compression for F12. | ||
== Owner == | == Owner == | ||
* Name: [[User:jnovy| Jindrich Novy]] [[User:notting| Bill Nottingham]] | * Name: [[User:jnovy| Jindrich Novy]], [[User:notting| Bill Nottingham]] | ||
* email: jnovy AT redhat DOT com, notting AT redhat DOT com | * email: jnovy AT redhat DOT com, notting AT redhat DOT com | ||
== Current status == | == Current status == | ||
* Targeted release: [[Releases/{{FedoraVersion||next}} | {{FedoraVersion|long|next}} ]] | * Targeted release: [[Releases/{{FedoraVersion||next}} | {{FedoraVersion|long|next}} ]] | ||
* Last updated: 2009- | * Last updated: 2009-08-19 | ||
* Percentage of completion: | * Percentage of completion: 100% (see Scope) | ||
<!-- CHANGE THE "FedoraVersion" TEMPLATES ABOVE TO PLAIN NUMBERS WHEN YOU COMPLETE YOUR PAGE. --> | <!-- CHANGE THE "FedoraVersion" TEMPLATES ABOVE TO PLAIN NUMBERS WHEN YOU COMPLETE YOUR PAGE. --> | ||
| Line 18: | Line 18: | ||
== Detailed Description == | == Detailed Description == | ||
RPM currently uses gzip payloads by default, with an option for bzip2. bzip2 adds greater compression, but with a large memory and cpu time | RPM currently uses gzip payloads by default, with an option for bzip2. bzip2 adds greater compression, but with a large memory and cpu time | ||
penalty. XZ allows better compression without these | penalty. XZ allows better compression without these penalties. | ||
== Benefit to Fedora == | == Benefit to Fedora == | ||
| Line 27: | Line 27: | ||
== Scope == | == Scope == | ||
# The xz package must be [http://bugzilla.redhat.com/show_bug.cgi?id=506339 reviewed] and added to active distributions - <b>DONE</b> | |||
# Rebuild RPM in F-12 to link against it - <b>DONE</b> | |||
# Rebuild RPM in F-11 to link against it (for upgrades) [https://admin.fedoraproject.org/updates/F11/FEDORA-2009-7986 F11 update] <b>DONE</b> | |||
4. Cherry-pick support from rpm-4.7.0 to rpm-4.6.1 | # [http://laiskiainen.org/rpm/patches/rpm-4.6.1-xz-support.patch Cherry-pick support from rpm-4.7.0 to rpm-4.6.1] - <b>DONE</b> | ||
# rebuild RPM 4.6.1 in F-10 to link against it (for upgrades) | |||
# rebuild RPM 4.6.1 for Koji to link against it (for the build system) - <b>DONE</b> | |||
# Change the default payload in redhat-rpm-config - <b>DONE</b> | |||
# Rebuild the distribution - <b>DONE</b> (in general) | |||
The final step is not required to support new payloads, but it is what provides the most benefit to Fedora users. | |||
== How To Test == | == How To Test == | ||
| Line 40: | Line 42: | ||
Check the dependencies of RPM itself, make sure it's linked against the XZ library, and that it provides the 'rpmlib(PayloadIsXz) dependency. | Check the dependencies of RPM itself, make sure it's linked against the XZ library, and that it provides the 'rpmlib(PayloadIsXz) dependency. | ||
Check successful extraction of RPM payload by installing or extracting contents of XZ compressed RPMs via rpm2cpio utility. | |||
Check that deltarpms can still be successfully created, and that the resulting RPM can be reassembled. | |||
== User Experience == | == User Experience == | ||
| Line 53: | Line 59: | ||
== Documentation == | == Documentation == | ||
In order to turn on XZ payload compression for binary RPMS one could define following macros in /usr/lib/rpm/macros globally or in ~/.rpmmacros file locally: | |||
<code>%_binary_payload w7.xzdio</code> | |||
This enables XZ payload compression level 7 (suggested default by upstream) for binary RPMs. This macro isn't defined by default and if undefined gzip compression is used. | |||
The <code>%_source_payload</code> macro can be used to change the payload compression for source RPMs. However, since source RPMs consist almost entirely of precompressed source archives, changing this has little effect, and will not be done for this feature. | |||
Note that currently you can use this configuration only with rpm-4.7.0 or higher which is present in Fedora 11 or rawhide. | |||
Also note that deltarpms between builds with different payload compression formats, while functional, usually save little bandwidth. | |||
== Release Notes == | == Release Notes == | ||
RPM in Fedora 12 has switched to using XZ compression (http://tukaani.org/xz/) in its packages | RPM in Fedora 12 has switched to using XZ compression (http://tukaani.org/xz/) in its packages to allow for better compression and smaller downloads for users. | ||
Users who wish to use | Users of older Fedora releases who wish to use RPMs built in Fedora 12 will need at least rpm-4.6.x from Fedora 10 or later. | ||
== Comments and Discussion == | == Comments and Discussion == | ||
* See [[Talk:Features/XZRpmPayloads]] | * See [[Talk:Features/XZRpmPayloads]] | ||
[[Category: | [[Category:FeatureAcceptedF12]] | ||
<!-- When your feature page is completed and ready for review --> | <!-- When your feature page is completed and ready for review --> | ||
<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler --> | <!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler --> | ||
Latest revision as of 16:36, 19 August 2009
XZ (LZMA) Payloads in RPM
Summary
We intend to switch RPM to use XZ (the new LZMA format) as default payload compression for F12.
Owner
- Name: Jindrich Novy, Bill Nottingham
- email: jnovy AT redhat DOT com, notting AT redhat DOT com
Current status
- Targeted release: Fedora 42
- Last updated: 2009-08-19
- Percentage of completion: 100% (see Scope)
Detailed Description
RPM currently uses gzip payloads by default, with an option for bzip2. bzip2 adds greater compression, but with a large memory and cpu time penalty. XZ allows better compression without these penalties.
Benefit to Fedora
- Smaller packages
- Smaller downloads
- More space on ISOs
- Smaller content for mirrors
Scope
- The xz package must be reviewed and added to active distributions - DONE
- Rebuild RPM in F-12 to link against it - DONE
- Rebuild RPM in F-11 to link against it (for upgrades) F11 update DONE
- Cherry-pick support from rpm-4.7.0 to rpm-4.6.1 - DONE
- rebuild RPM 4.6.1 in F-10 to link against it (for upgrades)
- rebuild RPM 4.6.1 for Koji to link against it (for the build system) - DONE
- Change the default payload in redhat-rpm-config - DONE
- Rebuild the distribution - DONE (in general)
The final step is not required to support new payloads, but it is what provides the most benefit to Fedora users.
How To Test
Query built rpms, make sure they are using the new payload format. This can be done by checking for the rpmlib(PayloadIsXz) dependency in the package itself.
Check the dependencies of RPM itself, make sure it's linked against the XZ library, and that it provides the 'rpmlib(PayloadIsXz) dependency.
Check successful extraction of RPM payload by installing or extracting contents of XZ compressed RPMs via rpm2cpio utility.
Check that deltarpms can still be successfully created, and that the resulting RPM can be reassembled.
User Experience
Users should see smaller packages and downloads. It should not be user-visible otherwise.
Dependencies
XZ package needs to be added to Fedora, and RPM built against it.
The XZ library is in beta, but the file format has been finalized (it is why we are using XZ instead of the not-finalized LZMA).
Contingency Plan
The contingency plan is to continue using gzipped payloads. If a mass rebuild has been done, another one would need done to rebuild with gzip payloads.
Documentation
In order to turn on XZ payload compression for binary RPMS one could define following macros in /usr/lib/rpm/macros globally or in ~/.rpmmacros file locally:
%_binary_payload w7.xzdio
This enables XZ payload compression level 7 (suggested default by upstream) for binary RPMs. This macro isn't defined by default and if undefined gzip compression is used.
The %_source_payload macro can be used to change the payload compression for source RPMs. However, since source RPMs consist almost entirely of precompressed source archives, changing this has little effect, and will not be done for this feature.
Note that currently you can use this configuration only with rpm-4.7.0 or higher which is present in Fedora 11 or rawhide.
Also note that deltarpms between builds with different payload compression formats, while functional, usually save little bandwidth.
Release Notes
RPM in Fedora 12 has switched to using XZ compression (http://tukaani.org/xz/) in its packages to allow for better compression and smaller downloads for users. Users of older Fedora releases who wish to use RPMs built in Fedora 12 will need at least rpm-4.6.x from Fedora 10 or later.
Comments and Discussion