LZMA for Live Images
Summary
Better compression technology (LZMA) allows us to ship more software on our live images.
Owner
- Name: Bruno Wolff III
- Email: <bruno@wolff.to>
Current status
- Targeted release: Fedora 14
- Last updated: 2010-02-14
- Percentage of completion: 30%
Detailed Description
LZMA compression typically provides better compression than zlib while still providing relatively fast uncompression. Once squashfs file systems are supported by the kernel (which is anticipated in 2.6.33) we can use them for building live images. It is anticipated that there will be savings that are significant for Fedora Live Images. (Once I start testing I'll have a few comparisons.) In particular the Games spin and the Desktop spins may be able to add back some desired packages cut for space reasons.
Benefit to Fedora
By allowing more packages to be put on a Live Image, we can make a few of our space constrained images better. For other images, having them be smaller will make them faster to download.
Scope
- Lougher's patches need to be accepted upstream. Right now they are in linux-next and will probably be in 2.6.34.
- Lougher's 4.1 (currently prerelease) version of squashfs-tools needs to be in the release.
- squashfs-tools 4.1 uses a wrapper that expects the lzma 4.65 SDK. Fedora doesn't have that. I need to understand it well enough to have it wrap the xz library instead. Updating the older lzma library in Fedora from 4.32 to 4.65 is a dead end, since it isn't developed upstream any more. Though maybe replacing it with just the current SDK library might make sense.
- Around 2010-02-05 Lougher added an xz wrapper to squashfs. This will probably be usable in Fedora without doing much work. It definitely builds, but I haven't tested it to make sure it works.
- squashfs-tools 4.1 uses a wrapper that expects the lzma 4.65 SDK. Fedora doesn't have that. I need to understand it well enough to have it wrap the xz library instead. Updating the older lzma library in Fedora from 4.32 to 4.65 is a dead end, since it isn't developed upstream any more. Though maybe replacing it with just the current SDK library might make sense.
- livecd-creator needs to be modified to have mksquashfs use lzma compression. (Probably as the default, with other compression methods specifiable as command line options.)
- If possible this should be usable significantly ahead of feature freeze so that spin owners can be pretty safe relying on it for their spins.
How To Test
- We need to make sure live images work.
- We need to make sure live images are smaller using LZMA compression.
- We need to make sure performance is acceptable. (I.e. that LZMA doesn't slow things down significantly on typical systems.)
User Experience
Live images will be smaller or have more included packages.
Dependencies
- squashfs-tools
- kernel (may need to ask for backport)
- livecd-tools
Contingency Plan
- Since the default in the new squashfs is too use zlib compression, we shouldn't have to revert squashfs-tools.
- livecd-creator would need to have at least some of the changes reverted.
Documentation
- Most likely livecd-creator would be getting a new option and would need its help updated along with the code changes.
Release Notes
- Mention the squashfs version change, that it supports LZMA compression, point to its documentation and note that it is supported in the kernel as well.
- Mention that livecd-creator uses LZMA by default, how to get zlib compression if desired for use with older kernels or other reasons.