Changes/FedoraCloudBtrfsByDefault

Make btrfs the default file system for Fedora Cloud

Summary

For cloud installs of Fedora, we want to provide advanced file system features to users in a transparent fashion. Thus, we are changing the file system for the Cloud Edition to Btrfs so we can leverage its features and capabilities to improve the quality of experience for Cloud users.

Owners

• Name: David Duncan, Chris Murphy, Josef Bacik, Michel Alexandre Salim, Davide Cavalca, Neal Gompa, Dusty Mabe, Matthew Almond
• Email: davdunc@amazon.com, chrismurphy@fedoraproject.org, josef@toxicpanda.com, michel@michel-slm.name, dcavalca@fb.com, ngompa13@gmail.com, dusty@dustymabe.com, malmond@fb.com
• Products: Fedora Cloud Edition
• Responsible WGs: Fedora Cloud WG

Current status

• Targeted release: Fedora Linux 35
• Last updated: 2021-06-15
• FESCo issue: #2617
• Tracker bug: #1972376
• Release notes tracker: #702

Detailed Description

Fedora Cloud Edition will switch to using Btrfs for its images. The configuration for the Cloud Edition will match the setup used on the desktop variants, as this has been very well-tested with production deployments across multiple Fedora Linux releases now.

This includes the same subvolume layout that is used on the desktop variants as introduced in Fedora Linux 33, as well as transparent Zstd compression as introduced in Fedora Linux 34.

Feedback

Userspace access options

Btrfs Cloud images will boot and function as normal even on hosts that do not support btrfs (e.g. RHEL). Resizing is also not impacted, as that's generally handled via cloud-init. However, the host will not be able to mount the image directly, nor read or write its contents. Libguestfs is also impacted, as it generally relies on an appliace built from the host kernel.

libguestfs container

A standalone Fedora-built libguestfs container will allow image access regardless of host kernel support. The container was approved in https://bugzilla.redhat.com/show_bug.cgi?id=1970071 and a preliminary version is already up at https://src.fedoraproject.org/container/libguestfs . Productionization is pending on container build issues in https://pagure.io/releng/issue/10161 being resolved.

grub2-mount

As grub can already read btrfs file systems, grub2-mount should be able to read only mount btrfs Cloud images. However, grub2-mount is currently disabled in CentOS Stream 8 (and hence in RHEL 8). We have filed https://bugzilla.redhat.com/show_bug.cgi?id=1970538 to get it enabled, and put up https://git.centos.org/rpms/grub2/pull-request/1 with the necessary changes to this end. CentOS Stream 9 already includes grub2-mount, so no change is needed there (nor should be needed for RHEL 9 as it inherits from CentOS Stream 9). Moreover, grub2 in CentOS Stream 8 currently does not include zstd support; this is being addressed in https://github.com/rhboot/grub2/pull/86.

btrfs-fuse

Because btrfs-progs already includes a full filesystem implementation in userspace, it should be possible to write a Fuse-based tool leveraging it to mount btrfs filesystems on any host with no kernel involvement. This has been confirmed as feasible, and a design is currently being put together.

Benefit to Fedora

The benefits are similar to the ones for Fedora desktop variants; however, there are specific benefits for Fedora Cloud:

• Adds support to Fedora Cloud for the Change to introduce support for Copy-on-Write enhancements to improve performance to package management
• Adds the ability to logically separate contents of the volume without dividing up the available space
  • Transparent compression: significantly reduces write amplification and improves effective I/O throughput
  • Reflinks and snapshots improve efficiency for use cases like containers (CRI-O, containerd, and Podman support both)
• Storage devices can be flaky, resulting in data corruption; Btrfs can help mitigate this
  • Everything is checksummed and verified on every read
  • Corrupt data results in EIO (input/output error), instead of resulting in application confusion, and isn’t replicated into backups and archives
• Improves system responsiveness under pressure
  • Btrfs has been tested in production to have proper IO isolation capability via cgroups2
  • Completes the resource control picture: memory, cpu, IO isolation
• File system resize
  • Online shrink and grow are cornerstones of the Btrfs design
• Complex storage setups are… complicated
  • Simple and comprehensive command interface. One master command
  • Simpler to boot, all code is in the kernel, no initramfs complexities
  • Simple and efficient file system replication, including incremental backups, with btrfs send and btrfs receive

Scope

• Proposal owners:
  • Submit PRs for Cloud Edition kickstarts to produce disk images using Btrfs.
• Release engineering: #10129
• Policies and guidelines: N/A
• Trademark approval: N/A

Upgrade/compatibility impact

Change will not affect upgrades.

How To Test

Once the change lands in Rawhide, spin up the images in AWS, GCP, and KVM/OpenStack to test to see systems boot and run.

User Experience

• Mostly transparent.
• Space savings and extend hardware life, via compression.
• Utilities for used and free space are expected to behave the same. No special commands are required.
• More detailed information can be revealed by btrfs specific commands.
• cp command will create reflink copies by default.

Dependencies

None.

Contingency Plan

• Contingency mechanism: Owner will revert changes back to the previous ext4 configuration
• Contingency deadline: Beta freeze
• Blocks release? Yes
• Blocks product? Cloud

Documentation

Strictly speaking, no extra documentation is required reading for users.

Release Notes

The default file system on the cloud is now Btrfs, following the desktop change in Fedora Linux 33. Fedora Server, IoT, and CoreOS are still specifically excluded.