Latest revision as of 04:42, 15 June 2020

Improve the Anaconda Btrfs preset

Summary

A few small changes will make it easier to do snapshots and rollbacks on btrfs, in the near future.

Owner

Name: Chris Murphy
Email: chrismurphy@fedoraproject.org
Product: Anaconda
Responsible WG: Workstation, KDE SIG

Current status

Targeted release: Fedora 33
Last updated: 2020-06-15
FESCo issue: <will be assigned by the Wrangler>
Tracker bug: <will be assigned by the Wrangler>
Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

/boot will be on btrfs. This is supported by GRUB for ~10 years, and recently in the installer. It will make snapshotting of /boot possible, to ensure parity with /usr on rollback.

Use subvolumes for the following mountpoints: /home /var/logs /var/lib/libvirt/images. This will make it possible to avoid snapshot and rollback of logs, VM images, and user home. NOTE: the design is WIP, but also is not user facing.

Prior work in this area: Revisiting How We Put Together Linux System containers/bubblewrap

Feedback

One big file system? What about clean installs and using /home?
Anaconda custom partitioning will let you reuse an existing 'home' subvolume to mount at '/home', without a reformat. A new 'root' subvolume must be created for '/' however. NOTE: Subvolumes are a bit special, but act mostly like directories, including having no size.

Benefit to Fedora

Optimize the 'btrfs preset' installation for today's use cases, and prepare for additional features. A significant benefit of btrfs is it can be a nearly drop-in replacement, and not burden users with learning esoteric things.

Default features:

One big file system, no longer run out of space on / or /home.
Full data integrity checking.ƒ
(Rootless) Podman will use snapshots.ƒ

Available optionally now, integration in the near future:

cgroupsv2 IO isolation, completes the resource control picture for better system responsiveness.ƒ
Transparent compression, file system/directory/file level granularity.ƒ
Snapshots and rollbacks, restore files using reflink copy.ƒ

ƒ These are btrfs only features.

Scope

Proposal owners:
- Implement /boot on dedicated ~1G btrfs volume with 'boot' subvolume
- Design a subvolume layout focusing on snapshot and rollback functionality, and modify the btrfs preset kickstart accordingly.

Other developers:
- Bootloader team to review and ack changes related to /boot.
- Anaconda to review and merge changes.
- Might affect dnf snapper plugin behavior.

Policies and guidelines: N/A

Trademark approval: N/A

Upgrade/compatibility impact

This change will not apply to upgrades.

How To Test

Install and use the system however you usually do, and it should behave the same.

User Experience

Transparent in normal use.
Always-on integrity verifies metadata and data. Corrupt data results in EIO, it never reaches user space. User is notified by kernel message about affected files by path.
Podman can use snapshots (default) or reflinks+overlayfs, at user discretion. Both are faster and more space efficient operations, than copy_up on ext4.

Users can opt-in to other btrfs features: snapshots, compression, cp --reflink efficient copies, online scrub, btrfs send/receive, at their own pace.

Dependencies

None

Contingency Plan

Contingency deadline: Revert all changes.
Blocks release? Yes.
Blocks product? Workstation/KDE are release blocking desktops.

Documentation

The user is not expected to know btrfs specific things for day to day use.

man 5 btrfs - for information about general btrfs information, mount options, and features.
man btrfs - for information about the btrfs command and sub commands. NOTE: Subcommands can be arbitrarily shortened, as long as it isn't ambiguous. 'btrfs fi us' is the same as 'btrfs filesystem usage'.

Users and developers alike, may be interested in:

Release Notes

@@ Line 5: / Line 5: @@
 == Summary ==
-A few small changes will make it easier to do snapshots and rollbacks on btrfs.
+A few small changes will make it easier to do snapshots and rollbacks on btrfs, in the near future.
 == Owner ==
@@ Line 15: / Line 15: @@
 * Product: Anaconda
 * Responsible WG: Workstation, KDE SIG
 == Current status ==
@@ Line 38: / Line 37: @@
 == Detailed Description ==
-System boot, root, and home will use btrfs for default installations using the Custom installation preset. [1]
+/boot will be on btrfs. This is supported by GRUB for ~10 years, and recently in the installer. It will make snapshotting of /boot possible, to ensure parity with /usr on rollback.
-Btrfs subvolumes mostly behave like directories. They have no free space size of their own, free space is shared with the file system. And yet, they can be snapshot, and be reused when reinstalling, e.g. it is possible keep an existing /home directory on clean install.
-Compared to the current preset, the change suggests adding subvolumes for the following mountpoints: <span style=color:blue>/etc /var /opt /root/root /srv /usr/local</span>
-[1] /boot on btrfs is supported by GRUB for ~10 years; enables snapshotting of /boot to ensure parity with /usr on rollback; needs design review and resource assessment.
+Use subvolumes for the following mountpoints: <span style=color:blue>/home /var/logs /var/lib/libvirt/images</span>. This will make it possible to avoid snapshot and rollback of logs, VM images, and user home. NOTE: the design is WIP, but also is not user facing.
+Prior work in this area:
+[http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html Revisiting How We Put Together Linux System]
+[https://github.com/containers/bubblewrap containers/bubblewrap]
 == Feedback ==
-Lorem ipsum ...
+''One big file system? What about clean installs and using /home?''</br >
+Anaconda custom partitioning will let you reuse an existing 'home' subvolume to mount at '/home', without a reformat. A new 'root' subvolume must be created for '/' however. NOTE: Subvolumes are a bit special, but act mostly like directories, including having no size.
 == Benefit to Fedora ==
-Make the default btrfs installation a little more optimized for today's use cases, and better prepared for future use cases. Btrfs makes advanced file system features available to user and developers for opt in usage, without requiring regular users to learn esoteric file system commands. e.g. <span style=color:red>df</span> will continue to correctly report used/free space. And no manual maintenance rituals are needed.
+Optimize the 'btrfs preset' installation for today's use cases, and prepare for additional features. A significant benefit of btrfs is it can be a nearly drop-in replacement, and not burden users with learning esoteric things.
-Features enabled by default:
+Default features:
-* File system metadata and data integrity checking
+* One big file system, no longer run out of space on / or /home.
-* Podman can use snapshots (default) or reflinks+overlayfs, at user discretion. Both are faster and more space efficient operations, than copy up on ext4.
+* Full data integrity checking.&fnof;
-* Flatpaks will be compressed by default.
+* (Rootless) Podman will use snapshots.&fnof;
-Optional features by CLI right now (future integration opportunities in GUI):
+Available optionally now, integration in the near future:
-* Enable/disable compression per file/directory (GNOME Files, KDE Konqueror)
+* cgroupsv2 IO isolation, completes the resource control picture for better system responsiveness.&fnof;
-* Online shrink and grow (GNOME Disks, KDE Partition Manager)
+* Transparent compression, file system/directory/file level granularity.&fnof;
-* Subvolumes, snapshots, restore files by reflink copy (GNOME Files, KDE Konqueror)
+* Snapshots and rollbacks, restore files using reflink copy.&fnof;
+&fnof; These are btrfs only features.
 == Scope ==
 * Proposal owners:
-** Implement /boot on dedicated ~1.5G btrfs volume with 'boot' subvolume
+** Implement /boot on dedicated ~1G btrfs volume with 'boot' subvolume
 ** Design a subvolume layout focusing on snapshot and rollback functionality, and modify the btrfs preset kickstart accordingly.
-** Enhance flatpak to set <span style=color:red>chattr +c</span> on system and user flatpak directories
 * Other developers:
-** Bootloader team to review and ack changes.
+** Bootloader team to review and ack changes related to /boot.
 ** Anaconda to review and merge changes.
-** Flatpak developers to review and merge above changes.
 ** Might affect dnf snapper plugin behavior.
@@ Line 94: / Line 92: @@
 * Transparent in normal use.
-* Corruption detected in data results in EIO. Corrupt data is never handed over to applications, unlike other file systems. Kernel messages will report an error and path to the affected file(s) so they can be replaced with known good copies.
+* Always-on integrity verifies metadata and data. Corrupt data results in EIO, it never reaches user space. User is notified by kernel message about affected files by path.
-* Better space utilization for containers and flatpaks.
+* Podman can use snapshots (default) or reflinks+overlayfs, at user discretion. Both are faster and more space efficient operations, than copy_up on ext4.
-* Users can opt-in to most btrfs features: snapshots, compression, <span style=color:red>cp --reflink</span> efficient copies, online scrub, btrfs send/receive
+* Users can opt-in to other btrfs features: snapshots, compression, <span style=color:red>cp --reflink</span> efficient copies, online scrub, btrfs send/receive, at their own pace.
 == Dependencies ==
@@ Line 116: / Line 115: @@
 * Users and developers alike, may be interested in:
-** [https://btrfs.wiki.kernel.org/index.php/Main_Page#Features additional features Expanded list of btrfs features]
+** [https://btrfs.wiki.kernel.org/index.php/Main_Page#Features Btrfs wiki: Full list of btrfs features]
-** [https://btrfs.wiki.kernel.org/index.php/Project_ideas btrfs project ideas]
+** [https://btrfs.wiki.kernel.org/index.php/Project_ideas Btrfs wiki: Project ideas]
+** [https://github.com/kdave/btrfs-progs/tree/master/libbtrfsutil btrfs-progs: libbtrfsutil]
 ** [https://github.com/knorrie/python-btrfs knorrie/python-btrfs]
-** [https://github.com/kdave/btrfs-progs/tree/master/libbtrfsutil libbtrfsutil]
 == Release Notes ==

Search

Changes/ImproveBtrfsPreset: Difference between revisions