From Fedora Project Wiki
mNo edit summary
 
(72 intermediate revisions by 7 users not shown)
Line 1: Line 1:
= Make btrfs the default file system for Workstation and KDE =
= Make btrfs the default file system for desktop variants =


== Summary ==
== Summary ==


LVM+ext4 has served us well, and it will remain available and supported for custom installs. The default file system for Workstation and KDE installs shall be btrfs.
For laptop and workstation installs of Fedora, we want to provide file system features to users in a transparent fashion. We want to add new features, while reducing the amount of expertise needed to deal with situations like [https://pagure.io/fedora-workstation/issue/152 running out of disk space.] Btrfs is well adapted to this role by design philosophy, let's make it the default.


== Owners ==
== Owners ==


* Names: [[User:Chrismurphy|Chris Murphy]], [[User:Ngompa|Neal Gompa]], [[User:Josef|Josef Bacik]], [[User:Salimma|Michel Alexandre Salim]], [[User:Dcavalca|Davide Cavalca]], [[User:eeickmeyer|Erich Eickmeyer]], [[User:ignatenkobrain|Igor Raits]]
* Name: [[User:Chrismurphy|Chris Murphy]], [[User:Ngompa|Neal Gompa]], [[User:Josef|Josef Bacik]], [[User:Salimma|Michel Alexandre Salim]], [[User:Dcavalca|Davide Cavalca]], [[User:eeickmeyer|Erich Eickmeyer]], [[User:ignatenkobrain|Igor Raits]], [[User:Raveit65|Wolfgang Ulbrich]], [[User:Zsun|Zamir SUN]], [[User:rdieter|Rex Dieter]], [[User:grinnz|Dan Book]], [[User:nonamedotc|Mukundan Ragavan]]
* Emails: chrismurphy@fedoraproject.org, ngompa13@gmail.com, josef@toxicpanda.com, michel@michel-slm.name, dcavalca@fb.com, erich@ericheickmeyer.com,ignatenkobrain@fedoraproject.org
* Email: chrismurphy@fedoraproject.org, ngompa13@gmail.com, josef@toxicpanda.com, michel@michel-slm.name, dcavalca@fb.com, erich@ericheickmeyer.com, ignatenkobrain@fedoraproject.org, fedora@raveit.de, zsun@fedoraproject.org, rdieter@gmail.com, grinnz@gmail.com, nonamedotc@gmail.com
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
-->
-->


* Product: Workstation, KDE, Fedora Jam
* Products: All desktop editions, spins, and labs
* Responsible WG: Workstation working group, and KDE SIG
* Responsible WGs: Workstation Working Group, KDE Special Interest Group


== Current status ==
== Current status ==
[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF33]]
[[Category:SystemWideChange]]
[[Category:SystemWideChange]]


Line 29: Line 29:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* FESCo issue: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/2429 #2429]
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1851166 #1851166]
* Release notes tracker: <will be assigned by the Wrangler>
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/530 #530]


== Detailed Description ==
== Detailed Description ==


The change is based on the installer's custom partitioning btrfs preset. It's been well tested for 7 years.
Fedora desktop edition/spin variants will switch to using Btrfs as the filesystem by default for new installs. Labs derived from these variants inherit this change, and other editions may opt into this change.
 
The change is based on the installer's custom partitioning Btrfs preset. It's been well tested for 7 years.


'''''Current partitioning'''''<br />
'''''Current partitioning'''''<br />
<code>vg/root</code> LV mounted at <code>/</code> and a <code>vg/home</code> LV mounted at <code>/home</code>. These are separate file system volumes, with separate free/used space.
<span style="color: tomato">vg/root</span> LV mounted at <span style="color: tomato">/</span> and a <span style="color: tomato">vg/home</span> LV mounted at <span style="color: tomato">/home</span>. These are separate file system volumes, with separate free/used space.


'''''Proposed partitioning'''''<br />
'''''Proposed partitioning'''''<br />
<code>root</code> subvolume mounted at <code>/</code> and <code>home</code> subvolume mounted at <code>/home</code>. Subvolumes don't have size, they act mostly like directories, space is shared.
<span style="color: tomato">root</span> subvolume mounted at <span style="color: tomato">/</span> and <span style="color: tomato">home</span> subvolume mounted at <span style="color: tomato">/home</span>. Subvolumes don't have size, they act mostly like directories, space is shared.


'''''Unchanged'''''<br />
'''''Unchanged'''''<br />
<code>/boot</code> will be a small ext4 volume. A separate boot is needed to boot dm-crypt sysroot installations; it's less complicated to keep the layout the same, regardless of whether sysroot is encrypted. There will be no automatic snapshots/rollbacks.
<span style="color: tomato">/boot</span> will be a small ext4 volume. A separate boot is needed to boot dm-crypt sysroot installations; it's less complicated to keep the layout the same, regardless of whether sysroot is encrypted. There will be no automatic snapshots/rollbacks.
 
If you select to encrypt your data, LUKS (dm-crypt) will be still used as it is today (with the small difference that Btrfs is used instead of LVM+Ext4). There is upstream work on getting native encryption for Btrfs that will be considered once ready and is subject of a different change proposal in a future Fedora release.


=== Optimizations (Optional) ===
=== Optimizations (Optional) ===


The detailed description above is the proposal. It's intended to be a minimalist and transparent switch. It's also the same as was proposed (and accepted) for Fedora 16. The following optimizations improve on the proposal, but are not critical. They are also transparent to most users. The general idea is agree to the base proposal first, and then consider these as enhancements.
The detailed description above is the proposal. It's intended to be a minimalist and transparent switch. It's also the same as was [[Features/F16BtrfsDefaultFs|proposed]] (and [https://lwn.net/Articles/446925/ accepted]) for Fedora 16. The following optimizations improve on the proposal, but are not critical. They are also transparent to most users. The general idea is agree to the base proposal first, and then consider these as enhancements. NOTE: These optimizations will not be used by default in Fedora 33, but are available for opt-in adoption and evaluation.


==== Boot on btrfs ====  
==== Boot on Btrfs ====  


* Instead of a 1G ext4 boot, create a 1G btrfs boot.  
* Instead of a 1G ext4 boot, create a 1G Btrfs boot.  
* Advantage: Makes it possible to include in a snapshot and rollback regime. GRUB has stable support for btrfs for 10+ years.  
* Advantage: Makes it possible to include in a snapshot and rollback regime. GRUB has stable support for Btrfs for 10+ years.  
* Scope: Contingent on bootloader and installer team review and approval.
* Scope: Contingent on bootloader and installer team review and approval. blivet should use <code>mkfs.btrfs --mixed</code>.


==== Compression ====
==== Compression ====


* Enable transparent compression using zstd on select directories: <code>/usr</code> <code>/var/lib/flatpak</code> <code>~/.local/share/flatpak</code>
* Enable transparent compression using zstd on select directories: <span style="color: tomato">/usr</span>,    <span style="color: tomato">/var/lib/flatpak</span>,    <span style="color: tomato">~/.local/share/flatpak</span>
* Advantage: Saves space and significantly reduces write amplification. May improve performance in some instances.
* Advantage: Saves space and significantly increase the lifespan of flash-based media by reducing write amplification. It may improve performance in some instances.
* Scope: Contingent on installer team review and approval to enhance anaconda to set the proper XATTR for each directory.
* Scope: Contingent on installer team review and approval to enhance anaconda to perform the installation using <code>mount -o compress=zstd</code>, then set the proper XATTR for each directory. The XATTR can't be set until after the directories are created via: rsync, rpm, or unsquashfs based installation.


==== Additional subvolumes ====
==== Additional subvolumes ====


* <code>/var/log/</code> <code>/var/lib/libvirt/</code> and <code>~/.local/share/gnome-boxes/images/</code> will use separate subvolumes.
* <span style="color: tomato">/var/log/</span>,    <span style="color: tomato">/var/lib/libvirt/images</span> and <span style="color: tomato">~/.local/share/gnome-boxes/images/</span> will use separate subvolumes.
* Advantage: Makes it easier to excluded them from snapshots, rollbacks, and send/receive.
* Advantage: Makes it easier to excluded them from snapshots, rollbacks, and send/receive. (Btrfs snapshotting is not recursive, it stops at a nested subvolume.)
* Scope: Anaconda knows how to do this already, just change the kickstart to add additional subvolumes, contingent on Anaconda review and approval.
* Scope: Anaconda knows how to do this already, just change the kickstart to add additional subvolumes (minus the subvolume in <span style="color: tomato">~/</span>. GNOME Boxes will need enhancement to detect that the user home is on Btrfs and create <span style="color: tomato">~/.local/share/gnome-boxes/images/</span> as a subvolume.


== Feedback ==
== Feedback ==


==== Red Hat doesn't support btrfs? Can Fedora do this? ====
==== Red Hat doesn't support Btrfs? Can Fedora do this? ====
 
Red Hat supports Fedora well, in many ways. But Fedora already works closely with, and depends on, upstreams. And this will be one of them. That's an important consideration for this proposal. The community has a stake in ensuring it is supported. Red Hat will never support Btrfs if Fedora rejects it. Fedora necessarily needs to be first, and make the persuasive case that it solves more problems than alternatives. Feature owners believe it does, hands down.
 
The Btrfs community has users that have been using it for most of the past decade at scale. It's been the default on openSUSE (and SUSE Linux Enterprise) since 2014, and Facebook has been using it for all their OS and data volumes, in their data centers, for almost as long. Btrfs is a mature, well-understood, and battle-tested file system, used on both desktop/container and server/cloud use-cases. We do have developers of the Btrfs filesystem maintaining and supporting the code in Fedora, one is a Change owner, so issues that are pinned to Btrfs can be addressed quickly.
 
==== What about device-mapper alternatives? ====
 
dm-thin (thin provisioning): [https://pagure.io/fedora-workstation/issue/152 Issue #152] still happens, because the installer won't over provision by default. It still requires manual intervention by the user to identify and resolve the problem. Upon growing a file system on dm-thin, the pool is over committed, and file system sizes become a fantasy: they don't add up to the total physical storage available. The truth of used and free space is only known by the thin pool, and CLI and GUI programs are unprepared for this. Integration points like rpm free space checks or GNOME disk-space warnings would have to be adapted as well.
 
dm-vdo: is not yet merged, and isn't as straightforward to selectively enable per directory and per file, as is the case on Btrfs using <code>chattr +c</code> on <span style="color: tomato">/var/lib/flatpaks/</span>.


Red Hat supports Fedora well, in many things. But Fedora already works closely with, and depends on, upstreams. And this will be one of them. That's an important consideration for this proposal. The community has a stake in ensuring it is supported. Red Hat will never support btrfs if Fedora rejects it. Fedora necessarily needs to be first, and make the persuasive case that it solve more problems than alternatives. Feature owners believe it does, hands down.
Btrfs solves the problems that need solving, with few side effects or pitfalls for users. It has more features we can take advantage of immediately and transparently: compression, integrity, and IO isolation. Many Btrfs features and optimizations can be opted into selectively per directory or file, such as compression and nodatacow, rather than as a layer that's either on or off.


==== What about UI/UX and integration in the desktop? ====
==== What about UI/UX and integration in the desktop? ====


If btrfs isn't the default file system, there's no commitment, nor reason to work on any UI/UX integration. There are ideas to make certain features discoverable. None are planned for Fedora 33. Anaconda already has sophisticated btrfs integration.
If Btrfs isn't the default file system, there's no commitment, nor reason to work on any UI/UX integration. There are ideas to make certain features discoverable: selective compression;  systemd-homed may take advantage of either Btrfs online resize, or near-term planned native encryption, which could make it possible to live convert non-encrypted homes to encrypted; and system snapshot and rollbacks.


==== What btrfs features are recommended and supported? ====
Anaconda already has sophisticated Btrfs integration.


This is the upstream [https://btrfs.wiki.kernel.org/index.php/Status btrfs feature status page]
==== What Btrfs features are recommended and supported? ====


Fedora is a community project. What is supported within Fedora depends on what the community decides to put forward in terms of resources.
The primary goal of this feature is to be largely transparent to the user. It does not require or expect users to learn new commands, or to engage in peculiar maintenance rituals.


When in doubt, use defaults. Be patient with yourself, and each other. There are few things you must learn about btrfs, but the toy box is full. It can be overwhelming. Features that sound familiar, like raid1, don't work the same as other implementations you're familiar with. There is lots of jargon. Take your time. No one needs to go from 0 kph to 100 kph overnight.
The full set of Btrfs features that is considered stable and enabled by default upstream will be enabled in Fedora. Fedora is a community project. What is supported within Fedora depends on what the community decides to put forward in terms of resources.


==== What is possible but not supported? ====
The upstream [https://btrfs.wiki.kernel.org/index.php/Status Btrfs feature status page].


No btrfs features will be disabled. The full box of toys is available. It is possible to get into trouble. You might also have more fun.
==== Are subvolumes really mostly like directories? ====
 
Subvolumes behave like directories in terms of navigation in both the GUI and CLI, e.g. <code>cp</code>, <code>mv</code>, <code>du</code>, owner/permissions, and SELinux labels. They also share space, just like a directory.
 
But it is an incomplete answer.
 
A subvolume is an independent file tree, with its own POSIX namespace, and has its own pool of inodes. This means inode numbers repeat themselves on a Btrfs volume. Inodes are only unique within a given subvolume. A subvolume has its own st_dev, so if you use <code>stat FILE</code> it reports a device value referring to the subvolume the file is in. And it also means hard links can't be created between subvolumes. From this perspective, subvolumes start looking more like a separate file system. But subvolumes share most of the other trees, so they're not truly independent file systems. They're also not block devices.


== Benefit to Fedora ==
== Benefit to Fedora ==


* problem
Problems Btrfs helps solve:
** btrfs helps solve


* Users running out of free space on either <code>/</code> or <code>/home</code> [https://pagure.io/fedora-workstation/issue/152 Workstation issue #152]
* Users running out of free space on either <span style="color: tomato">/</span> or <span style="color: tomato">/home</span> [https://pagure.io/fedora-workstation/issue/152 Workstation issue #152]
** "one big file system": no hard barriers like partitions or logical volumes
** "one big file system": no hard barriers like partitions or logical volumes
** transparent compression: significantly reduces write amplification, improves lifespan of storage hardware
** transparent compression: significantly reduces write amplification, improves lifespan of storage hardware
** reflinks and snapshots are more efficient for use cases like containers (Podman supports both)
** reflinks and snapshots are more efficient for use cases like containers (Podman supports both)
* Storage devices can be flaky, resulting in data corruption
** Everything is checksummed and verified on every read
** Corrupt data results in EIO (input/output error), instead of resulting in application confusion, and isn't replicated into backups and archives
* Poor desktop responsiveness when under pressure [https://pagure.io/fedora-workstation/issue/154 Workstation issue #154]
* Poor desktop responsiveness when under pressure [https://pagure.io/fedora-workstation/issue/154 Workstation issue #154]
** Currently only btrfs has proper IO isolation capability via cgroupsv2
** Btrfs has been tested in production to have proper IO isolation capability via cgroups2
** Completes the resource control picture: memory, cpu, IO isolation
** Completes the resource control picture: memory, cpu, IO isolation
* Storage devices betray users, resulting in data corruption
** Everything is checksummed and verified on every read
** Corrupt data results in EIO, instead of resulting in application confusion, and isn't replicated into backups and archives
* File system resize
* File system resize
** Online shrink and grow are fundamental to the design
** Whether provisioning an additional OS, or possible integration of systemd-homed, shrink capability is desired.
** Online shrink and grow are fundamental to Btrfs design
* Complex storage setups are... complicated
* Complex storage setups are... complicated
** Simple and comprehensive command interface. One master command
** Simple and comprehensive command interface. One master command
** Simpler to boot, all code is in the kernel, no initramfs complexities
** Simpler to boot, all code is in the kernel, no initramfs complexities
** Simple and efficient file system replication, including incremental backups, with <code>btrfs send</code> and <code>btrfs receive</code>


== Scope ==
== Scope ==
* Proposal owners:
* Proposal owners:
** Submit PR's for Anaconda to change <code>default_scheme = BTRFS</code> to the proper product files: Workstation, KDE.
** Submit PR's for Anaconda to change <code>default_scheme = BTRFS</code> to the proper product files. &radic;
** Multiple test days: build community support network
** Disk image variants of desktops will have their kickstarts adjusted to produce btrfs disk images. &radic;
** Aid with documentation
** The catch-all "Everything" netinstaller will use Btrfs by default. &radic;
** [https://pagure.io/fedora-qa/issue/634 Test days]: including building a community support network. &radic; (on-going)
** Aid with documentation (in-progress)
** Make btrfs in kernel built-in instead of a module (<code>CONFIG_BTRFS_FS=m</code> → <code>CONFIG_BTRFS_FS=y</code>) &radic;
** Bugs related to <code>btrfs</code> user-space commands should be filed against the <span style="color: blue">btrfs-progs</span> component. All other bugs should be filed against the <span style="color: blue">kernel</span> component, while also setting the Assignee field to: <span style="color: blue">fedora-kernel-btrfs@fedoraproject.org</span>


* Other developers:
* Other developers:
** Anaconda, review PRs and merge
** Anaconda, review PRs and merge
** Bootloader team, review PRs and merge
** Bootloader team, review PRs and merge
** Recommended optimization <code>chattr +C</code> set on the containing directory for virt-manager and GNOME Boxes.
** Recommended optimization <code>chattr +C</code> set on the containing directory for virt-manager and GNOME Boxes, via [https://www.redhat.com/archives/libvir-list/2020-July/msg01382.html libvirt] &radic;


* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Release engineering: [https://pagure.io/releng/issue/9545 #9545]


* Policies and guidelines: N/A
* Policies and guidelines: N/A
Line 132: Line 157:
Change will not affect upgrades.
Change will not affect upgrades.


Documentation will be provided for existing btrfs users to "retrofit" their setups to that of a default btrfs installations (base plus any approved options).
<s>Documentation will be provided for existing Btrfs users to "retrofit" their setups to that of a default Btrfs installation (base plus any approved options).</s> Layout has not been changed in Fedora 33.


== How To Test ==
== How To Test ==


'''''Today'''''<br />
'''''Today'''''<br />
Do a custom partitioning installation; change the scheme drop-down menu to btrfs; click the blue "automatically create partitions"; and install.<br />
Do a custom partitioning installation; change the scheme drop-down menu to Btrfs; click the blue "automatically create partitions"; and install.<br />
Fedora 31, 32, Rawhide, on x86_64 and ARM.
Fedora 31, 32, Rawhide, on x86_64 and ARM.


Line 147: Line 172:
==== Pros ====
==== Pros ====


* Mostly transparent
* Mostly transparent.
* Space savings from compression
* Users won't run out of space in / or /home as a result of competition for free space between them.
* Longer lifespan of hardware, also from compression.
* Space savings and extend hardware life, via compression.
* Utilities for used and free space, CLI and GUI, are expected to behave the same. No special commands are required.
* Utilities for used and free space, CLI and GUI, are expected to behave the same. No special commands are required.
* More detailed information can be revealed by <code>btrfs</code> specific commands.
* More detailed information can be revealed by <code>btrfs</code> specific commands.
* <code>cp</code> command will create reflink copies [https://src.fedoraproject.org/rpms/coreutils/c/5d08d14b/ by default.]


==== Enhancements opportunities ====
==== Enhancement opportunities ====
 
[https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/I3C5SECAXWNKMJSESZZOE35ZNM6MYL2W/ Nice UX if a MOTD popped up a link to support/recovery docs upon read-only boot.]


[https://bugzilla.redhat.com/show_bug.cgi?id=906591 updatedb does not index /home when /home is a bind mount] Also can affected rpm-ostree installations, including Silverblue.
[https://bugzilla.redhat.com/show_bug.cgi?id=906591 updatedb does not index /home when /home is a bind mount] Also can affected rpm-ostree installations, including Silverblue.


[https://gitlab.gnome.org/GNOME/gnome-usage/-/issues/49 GNOME Usage: Incorrect numbers when using multiple btrfs subvolumes] This isn't btrfs specific, happens with "one big ext4" volume as well.
[https://gitlab.gnome.org/GNOME/gnome-usage/-/issues/49 GNOME Usage: Incorrect numbers when using multiple btrfs subvolumes] This isn't Btrfs specific, happens with "one big ext4" volume as well.


[https://gitlab.gnome.org/GNOME/gnome-boxes/-/issues/88 GNOME Boxes, RFE: create qcow2 with 'nocow' option when on btrfs /home] This is btrfs specific, and is a recommended optimization for both GNOME Boxes and virt-manager.
[https://gitlab.gnome.org/GNOME/gnome-boxes/-/issues/88 GNOME Boxes, RFE: create qcow2 with 'nocow' option when on btrfs /home] This is Btrfs specific, and is a recommended optimization for both GNOME Boxes and virt-manager.


[https://github.com/containers/libpod/issues/6563 containers/libpod: automatically use btrfs driver if on btrfs]
[https://github.com/containers/libpod/issues/6563 containers/libpod: automatically use btrfs driver if on btrfs]
Line 177: Line 205:
== Documentation ==
== Documentation ==


Strictly speaking no documentation is required reading. But documentation will exist to help guide users as they dig beneath the minimal changes on the surface.
Strictly speaking no documentation is required reading for users. But there will be some Fedora documentation to help get the ball rolling.


For those who want to know more:
For those who want to know more:


[https://btrfs.wiki.kernel.org/index.php/Main_Page btrfs wiki main page]
[https://btrfs.wiki.kernel.org/index.php/Main_Page btrfs wiki main page and full feature list.]


<code>man 5 btrfs</code> contains: mount options, features, swapfile support, checksum algorithms, and more<br />
<code>man 5 btrfs</code> contains: mount options, features, swapfile support, checksum algorithms, and more<br />
Line 188: Line 216:




NOTE: The btrfs command will accept partially subcommand as long as it's not ambiguous. These are equivalent commands:<br />
NOTE: The btrfs command will accept an initial part of a subcommand name, as long as it is not ambiguous. These are equivalent commands:<br />
<code>btrfs subvolume snapshot</code><br />
<code>btrfs subvolume snapshot</code><br />
<code>btrfs sub snap</code><br />
<code>btrfs sub snap</code><br />
<code>btrfs su sn</code>
<code>btrfs su sn</code>


You'll discover your own convention. It might be preferable to write out the full command on forums and lists, but then maybe some folks don't learn about this shortcut?
You'll discover your own convention. It might be preferable to write out the full command on forums and lists, but then maybe some folks don't learn about this useful shortcut?


There will be some minimal Fedora documentation to help get the ball rolling.
For those who want to know a lot more:
 
[https://btrfs.wiki.kernel.org/index.php/Main_Page#Developer_documentation Btrfs developer documentation]<br />
[https://github.com/btrfs/btrfs-dev-docs/blob/master/trees.txt Btrfs trees]


== Release Notes ==
== Release Notes ==


The default file system is btrfs.
The default file system on the desktop is Btrfs. It specifically excludes Fedora Server, Cloud, IoT, and CoreOS. As a consequence of release engineering complexities, the seldom marketed "Everything" netinstaller will use Btrfs by default. It's advised that headless and PXE installation use cases should use the Fedora Server netinstaller.
 
All installer images continue to have a custom partitioning option, enabling users to choose their preferred layout and file system.

Latest revision as of 04:22, 3 February 2021

Make btrfs the default file system for desktop variants

Summary

For laptop and workstation installs of Fedora, we want to provide file system features to users in a transparent fashion. We want to add new features, while reducing the amount of expertise needed to deal with situations like running out of disk space. Btrfs is well adapted to this role by design philosophy, let's make it the default.

Owners

  • Products: All desktop editions, spins, and labs
  • Responsible WGs: Workstation Working Group, KDE Special Interest Group

Current status

Detailed Description

Fedora desktop edition/spin variants will switch to using Btrfs as the filesystem by default for new installs. Labs derived from these variants inherit this change, and other editions may opt into this change.

The change is based on the installer's custom partitioning Btrfs preset. It's been well tested for 7 years.

Current partitioning
vg/root LV mounted at / and a vg/home LV mounted at /home. These are separate file system volumes, with separate free/used space.

Proposed partitioning
root subvolume mounted at / and home subvolume mounted at /home. Subvolumes don't have size, they act mostly like directories, space is shared.

Unchanged
/boot will be a small ext4 volume. A separate boot is needed to boot dm-crypt sysroot installations; it's less complicated to keep the layout the same, regardless of whether sysroot is encrypted. There will be no automatic snapshots/rollbacks.

If you select to encrypt your data, LUKS (dm-crypt) will be still used as it is today (with the small difference that Btrfs is used instead of LVM+Ext4). There is upstream work on getting native encryption for Btrfs that will be considered once ready and is subject of a different change proposal in a future Fedora release.

Optimizations (Optional)

The detailed description above is the proposal. It's intended to be a minimalist and transparent switch. It's also the same as was proposed (and accepted) for Fedora 16. The following optimizations improve on the proposal, but are not critical. They are also transparent to most users. The general idea is agree to the base proposal first, and then consider these as enhancements. NOTE: These optimizations will not be used by default in Fedora 33, but are available for opt-in adoption and evaluation.

Boot on Btrfs

  • Instead of a 1G ext4 boot, create a 1G Btrfs boot.
  • Advantage: Makes it possible to include in a snapshot and rollback regime. GRUB has stable support for Btrfs for 10+ years.
  • Scope: Contingent on bootloader and installer team review and approval. blivet should use mkfs.btrfs --mixed.

Compression

  • Enable transparent compression using zstd on select directories: /usr, /var/lib/flatpak, ~/.local/share/flatpak
  • Advantage: Saves space and significantly increase the lifespan of flash-based media by reducing write amplification. It may improve performance in some instances.
  • Scope: Contingent on installer team review and approval to enhance anaconda to perform the installation using mount -o compress=zstd, then set the proper XATTR for each directory. The XATTR can't be set until after the directories are created via: rsync, rpm, or unsquashfs based installation.

Additional subvolumes

  • /var/log/, /var/lib/libvirt/images and ~/.local/share/gnome-boxes/images/ will use separate subvolumes.
  • Advantage: Makes it easier to excluded them from snapshots, rollbacks, and send/receive. (Btrfs snapshotting is not recursive, it stops at a nested subvolume.)
  • Scope: Anaconda knows how to do this already, just change the kickstart to add additional subvolumes (minus the subvolume in ~/. GNOME Boxes will need enhancement to detect that the user home is on Btrfs and create ~/.local/share/gnome-boxes/images/ as a subvolume.

Feedback

Red Hat doesn't support Btrfs? Can Fedora do this?

Red Hat supports Fedora well, in many ways. But Fedora already works closely with, and depends on, upstreams. And this will be one of them. That's an important consideration for this proposal. The community has a stake in ensuring it is supported. Red Hat will never support Btrfs if Fedora rejects it. Fedora necessarily needs to be first, and make the persuasive case that it solves more problems than alternatives. Feature owners believe it does, hands down.

The Btrfs community has users that have been using it for most of the past decade at scale. It's been the default on openSUSE (and SUSE Linux Enterprise) since 2014, and Facebook has been using it for all their OS and data volumes, in their data centers, for almost as long. Btrfs is a mature, well-understood, and battle-tested file system, used on both desktop/container and server/cloud use-cases. We do have developers of the Btrfs filesystem maintaining and supporting the code in Fedora, one is a Change owner, so issues that are pinned to Btrfs can be addressed quickly.

What about device-mapper alternatives?

dm-thin (thin provisioning): Issue #152 still happens, because the installer won't over provision by default. It still requires manual intervention by the user to identify and resolve the problem. Upon growing a file system on dm-thin, the pool is over committed, and file system sizes become a fantasy: they don't add up to the total physical storage available. The truth of used and free space is only known by the thin pool, and CLI and GUI programs are unprepared for this. Integration points like rpm free space checks or GNOME disk-space warnings would have to be adapted as well.

dm-vdo: is not yet merged, and isn't as straightforward to selectively enable per directory and per file, as is the case on Btrfs using chattr +c on /var/lib/flatpaks/.

Btrfs solves the problems that need solving, with few side effects or pitfalls for users. It has more features we can take advantage of immediately and transparently: compression, integrity, and IO isolation. Many Btrfs features and optimizations can be opted into selectively per directory or file, such as compression and nodatacow, rather than as a layer that's either on or off.

What about UI/UX and integration in the desktop?

If Btrfs isn't the default file system, there's no commitment, nor reason to work on any UI/UX integration. There are ideas to make certain features discoverable: selective compression; systemd-homed may take advantage of either Btrfs online resize, or near-term planned native encryption, which could make it possible to live convert non-encrypted homes to encrypted; and system snapshot and rollbacks.

Anaconda already has sophisticated Btrfs integration.

What Btrfs features are recommended and supported?

The primary goal of this feature is to be largely transparent to the user. It does not require or expect users to learn new commands, or to engage in peculiar maintenance rituals.

The full set of Btrfs features that is considered stable and enabled by default upstream will be enabled in Fedora. Fedora is a community project. What is supported within Fedora depends on what the community decides to put forward in terms of resources.

The upstream Btrfs feature status page.

Are subvolumes really mostly like directories?

Subvolumes behave like directories in terms of navigation in both the GUI and CLI, e.g. cp, mv, du, owner/permissions, and SELinux labels. They also share space, just like a directory.

But it is an incomplete answer.

A subvolume is an independent file tree, with its own POSIX namespace, and has its own pool of inodes. This means inode numbers repeat themselves on a Btrfs volume. Inodes are only unique within a given subvolume. A subvolume has its own st_dev, so if you use stat FILE it reports a device value referring to the subvolume the file is in. And it also means hard links can't be created between subvolumes. From this perspective, subvolumes start looking more like a separate file system. But subvolumes share most of the other trees, so they're not truly independent file systems. They're also not block devices.

Benefit to Fedora

Problems Btrfs helps solve:

  • Users running out of free space on either / or /home Workstation issue #152
    • "one big file system": no hard barriers like partitions or logical volumes
    • transparent compression: significantly reduces write amplification, improves lifespan of storage hardware
    • reflinks and snapshots are more efficient for use cases like containers (Podman supports both)
  • Storage devices can be flaky, resulting in data corruption
    • Everything is checksummed and verified on every read
    • Corrupt data results in EIO (input/output error), instead of resulting in application confusion, and isn't replicated into backups and archives
  • Poor desktop responsiveness when under pressure Workstation issue #154
    • Btrfs has been tested in production to have proper IO isolation capability via cgroups2
    • Completes the resource control picture: memory, cpu, IO isolation
  • File system resize
    • Whether provisioning an additional OS, or possible integration of systemd-homed, shrink capability is desired.
    • Online shrink and grow are fundamental to Btrfs design
  • Complex storage setups are... complicated
    • Simple and comprehensive command interface. One master command
    • Simpler to boot, all code is in the kernel, no initramfs complexities
    • Simple and efficient file system replication, including incremental backups, with btrfs send and btrfs receive

Scope

  • Proposal owners:
    • Submit PR's for Anaconda to change default_scheme = BTRFS to the proper product files. √
    • Disk image variants of desktops will have their kickstarts adjusted to produce btrfs disk images. √
    • The catch-all "Everything" netinstaller will use Btrfs by default. √
    • Test days: including building a community support network. √ (on-going)
    • Aid with documentation (in-progress)
    • Make btrfs in kernel built-in instead of a module (CONFIG_BTRFS_FS=m → CONFIG_BTRFS_FS=y) √
    • Bugs related to btrfs user-space commands should be filed against the btrfs-progs component. All other bugs should be filed against the kernel component, while also setting the Assignee field to: fedora-kernel-btrfs@fedoraproject.org
  • Other developers:
    • Anaconda, review PRs and merge
    • Bootloader team, review PRs and merge
    • Recommended optimization chattr +C set on the containing directory for virt-manager and GNOME Boxes, via libvirt √
  • Release engineering: #9545
  • Policies and guidelines: N/A
  • Trademark approval: N/A

Upgrade/compatibility impact

Change will not affect upgrades.

Documentation will be provided for existing Btrfs users to "retrofit" their setups to that of a default Btrfs installation (base plus any approved options). Layout has not been changed in Fedora 33.

How To Test

Today
Do a custom partitioning installation; change the scheme drop-down menu to Btrfs; click the blue "automatically create partitions"; and install.
Fedora 31, 32, Rawhide, on x86_64 and ARM.

Once change lands
It should be simple enough to test, just do a normal install.

User Experience

Pros

  • Mostly transparent.
  • Users won't run out of space in / or /home as a result of competition for free space between them.
  • Space savings and extend hardware life, via compression.
  • Utilities for used and free space, CLI and GUI, are expected to behave the same. No special commands are required.
  • More detailed information can be revealed by btrfs specific commands.
  • cp command will create reflink copies by default.

Enhancement opportunities

Nice UX if a MOTD popped up a link to support/recovery docs upon read-only boot.

updatedb does not index /home when /home is a bind mount Also can affected rpm-ostree installations, including Silverblue.

GNOME Usage: Incorrect numbers when using multiple btrfs subvolumes This isn't Btrfs specific, happens with "one big ext4" volume as well.

GNOME Boxes, RFE: create qcow2 with 'nocow' option when on btrfs /home This is Btrfs specific, and is a recommended optimization for both GNOME Boxes and virt-manager.

containers/libpod: automatically use btrfs driver if on btrfs

Dependencies

None.

Contingency Plan

  • Contingency mechanism: Owner will revert changes back to LVM+ext4
  • Contingency deadline: Beta freeze
  • Blocks release? Yes
  • Blocks product? Workstation and KDE

Documentation

Strictly speaking no documentation is required reading for users. But there will be some Fedora documentation to help get the ball rolling.

For those who want to know more:

btrfs wiki main page and full feature list.

man 5 btrfs contains: mount options, features, swapfile support, checksum algorithms, and more
man btrfs contains an overview of the btrfs subcommands
man btrfs <subcommand> will show the man page for that subcommand


NOTE: The btrfs command will accept an initial part of a subcommand name, as long as it is not ambiguous. These are equivalent commands:
btrfs subvolume snapshot
btrfs sub snap
btrfs su sn

You'll discover your own convention. It might be preferable to write out the full command on forums and lists, but then maybe some folks don't learn about this useful shortcut?

For those who want to know a lot more:

Btrfs developer documentation
Btrfs trees

Release Notes

The default file system on the desktop is Btrfs. It specifically excludes Fedora Server, Cloud, IoT, and CoreOS. As a consequence of release engineering complexities, the seldom marketed "Everything" netinstaller will use Btrfs by default. It's advised that headless and PXE installation use cases should use the Fedora Server netinstaller.

All installer images continue to have a custom partitioning option, enabling users to choose their preferred layout and file system.

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/BtrfsByDefault&oldid=603236"