Changes/BtrfsTransparentCompression

Enable btrfs transparent zstd compression by default

Summary

On variants using btrfs as the default filesystem, enable transparent compression using zstd. Compression saves space and can significantly increase the lifespan of flash-based media by reducing write amplification. It can also increase read and write performance.

Owners

• Name: Michel Salim, Davide Cavalca, Josef Bacik
• Email: michel@michel-slm.name, dcavalca@fb.com, josef@toxicpanda.com

Current status

• Targeted release: Fedora 34
• Last updated: 2021-01-15
• FESCo issue: #2538
• Tracker bug: #1916918
• Release notes tracker: #635

Detailed description

Transparent compression is a btrfs feature that allows a btrfs filesystem to apply compression on a per-file basis. Of the three supported algorithms, zstd is the one with the best compression speed and ratio. Enabling compression saves space, but it also reduces write amplification, which is important for SSDs. Depending on the workload and the hardware, compression can also result in an increase in read and write performance.

See https://pagure.io/fedora-btrfs/project/issue/5 for details. This was originally scoped as an optimization for https://fedoraproject.org/wiki/Changes/BtrfsByDefault during Fedora 33.

Feedback

Why use zstd:1 specifically?

https://hackmd.io/kIMJv7yHSiKoAq1MPcCMdw has an analysis of the various compression levels and their impact, both in terms of CPU usage and disk space savings.

Will /boot be compressed?

By default we still put /boot on ext4 (though /boot on btrfs is possible with advanced partitioning). GRUB has support for zstd-compressed Btrfs partitions since 2018 so this should work just fine.

We plan a forthcoming change once Changes/UnifyGrubConfig lands to make sure that the GRUB environment is stored in a partition that it can write to.

Currently the hidden GRUB menu in Fedora Workstation (Changes/HiddenGrubMenu, Changes/CleanupGnomeHiddenBootMenuIntegration) will not work on BIOS-based systems; after UnifyGrubConfig, this might not work even on EFI systems, so we are not recommending /boot on btrfs for now because of this regression.

Benefit to Fedora

Better disk space usage, reduction of write amplification, which in turn helps increase lifespan and performance on SSDs and other flash-based media. It can also increase read and write performance.

Scope

• Proposal owners:
  • Update anaconda to perform the installation using mount -o compress=zstd:1
  • Set the proper option in fstab (alternatively: set the XATTR)
  • Update disk image build tools to enable compression:
    • lorax
    • appliance-tools
    • osbuild
    • imagefactory
  • [optional] Add support for setting compression level when defragmenting
  • [optional] Add support for setting compression level using btrfs property
• Other developers:
  • anaconda: review PRs as needed
• Release engineering: https://pagure.io/releng/issue/9920
• Policies and guidelines: N/A
• Trademark approval: N/A

Upgrade/compatibility impact

This Change only applies to newly installed systems. See How to test for converting an existing system.

How to test

Existing systems can be converted to use compression manually with btrfs filesystem defrag -czstd -r, updating /etc/fstab to add the compress=zstd:1 mount option and remounting (mount -o remount /)

User experience

Compression will result in file sizes (e.g. as reported by du) not matching the actual space occupied on disk. The compsize utility can be used to examine the compression type, effective compression ration and actual size.

Dependencies

Anaconda will need to be updated to perform the installation using mount -o compress=zstd:1

Contingency plan

• Contingency mechanism: will not include PR patches if not merged upstream and will not enable
• Contingency deadline: Final freeze
• Blocks release? No
• Blocks product? No

Documentation

https://btrfs.wiki.kernel.org/index.php/Compression

Release Notes

Transparent compression of the filesystem using zstd is now enabled by default. Use the compsize utility to find out the actual size on disk of a given file.