Features/UsrMove

Move all to /usr

Summary

Provide a simple way of mounting almost the entire installed operating system read-only, atomically snapshot it, or share it between multiple hosts to save maintenance and space. Instead of spreading RPM package content all over the place in the filesystem, and artificially separate /bin from /usr/bin and /lib from /usr/lib, move all content to /usr and provide only symlinks in the root filesystem.

/usr on its own filesystem provides a lot of valuable options in custom setups. For historic reasons, we split-off more and more tools from /usr and put them in /. But, advanced features in today's systems can not really bootup with an empty /usr anymore. More and more fails in subtle ways in such setups.

Instead of moving more tools to /, we today already require /usr to be mounted from inside the initramfs, to be available before the real 'init' starts. The split of the root filesystem an /usr serves no purpose in Linux anymore and only complicates or prevents simple and more flexible setups.

Owner

• Name: Harald Hoyer
• Email: harald@redhat.com
• Name: Kay Sievers
• Email: kay@redhat.com

Current status

• Targeted release: Fedora 17
• Last updated: 2011-09-21
• Percentage of completion: 5%

Detailed Description

There is no way to reliably bring up a modern system with an empty /usr, there are two alternatives to fix it: copy /usr back to the rootfs or use an initramfs which can hide the split-off from the system.

Historically /bin, /sbin, /lib had the purpose to contain the utilities to mount /usr. This role can now be taken by the initramfs. Because the initramfs knows, where to find the root partition (which includes /etc), it can parse /etc/fstab and other configuration files and mount /usr before it finally switches the root partition and executes /usr/bin/init. From this point on init mounts the remaining partitions in /etc/fstab and the system starts as usual.

The long-term plan is to clean up the mess and confusion the current split of / vs. /usr has created. All tools will move back to /usr where they belong, and the rootfs will only contain compat-symlinks into /usr. Almost the entire system installed by packages will reside in /usr. This will split all non-host specific data to /usr. /usr can then be seen as the Unix System Resources partition (/System), which defines the base operating system (e.g. F18 or RHEL-7).

This new /usr could be mounted read-only by default, while the rootfs is read-write and contains only empty mount points, compat-symlinks to /usr and the host-specific data like /etc, /root, /srv. Compared to today's setups, the rootfs will be very small. The new /usr could also easily be shared read-only across several systems, and it would contain almost the entire system. Such setups are more efficient, can optionally provide a lot more security, are more flexible, provide more sane options for custom setups, and are much simpler to setup and maintain.

This leaves us with the following well-defined directories, which compose the base of the system:

• /usr - installed system; shareable; possibly read-only
• /etc - config data; non-shareable
• /var - persistent data; non-shareable;
• /run - volatile data; non-shareable; mandatory tmpfs filesystem

In the process of moving /bin and /sbin to /usr/bin, /usr/sbin can be moved also to /usr/bin.

/
|-- etc
|-- usr
|   |-- bin
|   |-- lib
|   `-- lib64
|-- run
|-- var
|-- bin -> usr/bin
|-- sbin -> usr/bin
|-- lib -> usr/lib
`-- lib64 -> usr/lib64

Benefit to Fedora

• Clear separation of operating system and host specific resources.
• /usr can be read-only and shareable.
• /usr can be easily snapshotted.

Scope

The ability to share /usr is especially useful for clusters and virtual machines. The ability to mount /usr read-only (e.g. on read-only media) adds to the security of the machine.

User Experience

• less toplevel directories

Dependencies

• initramfs (dracut)
• changes in selinux policies
• repackaging of packages with content in /bin, /sbin, /lib*
• drop consolehelper to move /usr/sbin/* to /usr/bin
• alternatives symlinks?
• filesystem rpm, toplevel symlinks

Roadmap

• Begin changing rpm packages with files in /bin, /sbin, /usr/sbin, /lib, /lib64.
• Make backward compat symlinks in %post and %ghost those symlinks:

%post
# create compat symlink for tools as long as root directories are not converted to symlinks
if ! test -L /bin; then
    ln -s ../usr/bin/foo /bin/foo
    ln -s ../usr/bin/bar /bin/bar
fi
if ! test -L /sbin; then
    ln -s ../usr/bin/buz /sbin/buz
fi

%files
%ghost %attr(777, root, root) /bin/foo
%ghost %attr(777, root, root) /bin/bar
%ghost %attr(777, root, root) /sbin/buz

• RPM: 257 packages that install files in the root filesystem.
• Change SELinux policies.
• On new installation: create symlinks /bin -> usr/bin, /sbin -> usr/bin, /lib -> usr/lib, /lib64 -> usr/lib64, /usr/sbin -> bin. These links will take care that installed packages do not install compat symlinks in %post.
• Make sure dracut is able to mount needed filesystems specifies in /etc/fstab before starting systemd.
• Drop consolehelper to enable the /usr/sbin -> /usr/bin move.

Contingency Plan

• We do not support to bootup with an empty /usr today, so moving things to /usr and have compat links in the rootfs should be low risk. If things turn out to get difficult, we can delay the creation of the /bin /sbin /lib lib64 compat links to a later release.

Comments and Discussion

• See Talk:Features/UsrMove