Changes/LibvirtModularDaemons

Libvirt Modular Daemons

Summary

Historically all libvirt functionality was provided in the monolithic libvirtd daemon. Upstream has developed a new modular architecture for libvirt where each driver is run in its own daemon. Primarily this provides better robustness as a flaw in the node device daemon will not affect the QEMU daemon and vica-versa. It should also have slightly lower host startup overhead, because only the installed hypervisor daemon(s) will need to be fully started on boot, the other daemons can be socket activated on demand.

Owner

• Name: Daniel Berrange
• Email: berrange@redhat.com

Current status

• Targeted release: Fedora Linux 35
• Last updated: 2021-06-10
• FESCo issue: <will be assigned by the Wrangler>
• Tracker bug: <will be assigned by the Wrangler>
• Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

Historically all libvirt functionality was provided in the monolithic libvirtd daemon. Upstream has developed a new modular architecture for libvirt where each driver is run in its own daemon. Daemons include virtqemud, virtlxcd, virtnetworkd, virtnodedevd, virtsecretd, etc, one for each libvirt stateful hypervisor driver and secondary support driver. The new daemons listen on new socket paths and only support UNIX sockets. For remote off-host TCP/TLS access and for backwards compatibility with clients expecting the libvirtd UNIX socket paths, a virtproxyd daemon is also provided. The latter accepts RPC requests and forwards to the appropriate modular daemon. Further information is available in the Upstream documentation

The work will primarily involve changes in the systemd presets related to libvirt. Currently the Fedora systemd presets enable libvirtd.service. While libvirtd does support socket activation this is not used during host startup, because the libvirtd daemon needs to be fully started in order to auto-start QEMU guests that may be configured. The current libvirtd.service uses a timeout, however, that causes libvirtd to shutdown after 2 minutes if no VM is running.

This requirement for fully starting the service on boot still exists in the modular daemons, however, we do not need to start all the modular daemons on boot. It is sufficient to enable just the service units for the hypervisor drivers QEMU, Xen and LXC. These will again this will use a 2 minute timeout so they shut down if no guest/container are running. Further, a default install of libvirt in Fedora is only expected to provide the QEMU libvirt daemon.

The remaining modular daemons for libvirt non-hypervisor drivers can exclusively rely on socket activation, because while they do provide autostart capabilities for non-VM resources, these resources are only needed when a VM actually starts.

The current '/usr/lib/systemd/system-preset/90-default.preset' config has

 libvirtd.service

and due to dependencies, this implies libvirtd.socket, libvirtd-ro.socket, libvirtd-admin.socket, virtlogd.socket, virtlockd.socket also get enabled by default

The intended change is to remove libvirtd.service and instead add

 virtqemud.service
 virtxend.service
 virtlxcd.service

 virtinterfaced.socket
 virtnetworkd.socket
 virtnodedevd.socket
 virtnwfilterd.socket
 virtproxyd.socket
 virtsecretd.socket
 virtstoraged.socket

Feedback

TBD

Benefit to Fedora

The primary benefit of the modular daemons is robustness of the management stack. There are times when libvirt code has exhibited hangs or crashes. By having one daemon per libvirt driver, the impact of these bugs is limited. The most important benefit of this is that a bug in the node device / storage daemons will not impact management of any running QEMU VMs.

The secondary benefit of the modular daemons is improved host startup overhead. When the libvirtd daemon starts, all drivers are initialized and this can take a significant amount of CPU time for the node device daemon in particular as it extracts information about all host hardware. Reducing the amount of functionality

The third potential future benefit is that it enables development of more useful SELinux policy. The current policy for libvirtd is no better than unconfined due to the broad range of features that must be allowed. At least some of the modular daemons can potentially have useful SELinux policy written. Note there is no active work on this aspect.

Scope

• Proposal owners:

Rebuild libvirt to prefer connecting to modular daemons by default

Submit a patch updating 90-defaults.preset in the fedora-release package to enable modular daemons by default

• Other developers:

Review the proposal for fedora-release preset change

• Release engineering: N/A

• Policies and guidelines: N/A

• Trademark approval: N/A

• Alignment with Objectives: N/A

Upgrade/compatibility impact

Existing Fedora installs using libvirtd will be unaffected by the change in presets and will continue to use monolithic libvirtd. To opt-in to using the new modular daemons follow instructions at https://libvirt.org/daemons.html#switching-to-modular-daemons

New Fedora installs will get modular daemons instead of libvirtd. This is functionally transparent to applications using libvirt APIs.

This may have an impact on automation tools such as ansible / puppet which have been written to manage libvirtd. These will need adapting to manage the new modular daemons, or the host can be re-configured to use libvirtd again in the short term by applying the inverse of the logic at https://libvirt.org/daemons.html#switching-to-modular-daemons

Administrators who wish to enable TCP/TLS sockets for remote access to libvirt, for the sake of migration, will need to configure this using the virtproxyd daemon instead of libvirtd.

At some point in the future it is hoped that the monolithic libvirtd will be entirely removed by libvirt upstream. This is predicated on prolonged successful deployment of the modular daemons in one or more major distros. Thus the likely timeframe is Fedora 37 at the earliest (i.e. min 1 year from successful usage of modular daemons in a distro).

How To Test

Hardware: No special hardware is required for testing, since QEMU can run in emulated mode if hardware virtualization is not available

Preparation: Deploy a completely fresh Fedora install, since systemd preset changes only take effect on initial installation. Ensure that virtualization is installed with QEMU/KVM.

Actions: Boot the system and observice that 'virtqemud' is running immediately on completion of boot up with an arg of '--timeout=120'.

After two minutes the daemon should exit, and will automatically restart if 'virsh -c qemu:///system list' is run as root.

Deploy a new virtual machine using virt-install / virt-manager.

Set the new VM to autostart, eg 'virsh -c qemu:///system autostart $GUESTNAME'

Shutdown the guest and reboot the host

Upon completion of boot up observe that 'virtqemud' is running, and also a QEMU guest is running. 'virtqemud' should remain running for as long as the guest is running.

The 'virtnetworkd' will also be running if the guest is connected to the 'virbr0' virtual network device provided by libvirt.

User Experience

Users of libvirt applications should not see any functional changes in their experience, since the new modular daemons are intended to be fully backwards compatible with existing libvirtd daemon.

Administrators configuring hosts, however, will need to be aware that a different set of libvirt daemons will be running than they have seen in the past. See upgrade impact section for likely impact on administrators experience.

There may be a small improvement in boot time performance when libvirt is installed.

Dependencies

The fedora-release package needs updating with the new systemd presets.

Contingency Plan

• Contingency mechanism:

Should unexpected problems with the change arise, the changes in libvirt and fedora-release will be reverted to the historical known good behaviour of libvirt in previous Fedora.

• Contingency deadline: Beta freeze

• Blocks release? Yes

Documentation

Primary upstream guidance is https://libvirt.org/daemons.html

N/A (not a System Wide Change)

Release Notes

The libvirt package has switched to using modular daemons by default. The monolithic libvirtd daemon will no longer be started. In its place a hypervisor specific daemon will be started, along with one or more secondary driver daemons.

Applications using libvirt APIs should not notice any difference in behaviour.

Administrators configuring a host manually, or using automation tools such as ansible/puppet, may need to take account of the new daemons that are providing libvirt functionality.