Changes/ReplaceDnfWithDnf5

Make DNF5 The Default

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Make DNF5 the new default packaging tool. The change will replace DNF, YUM, and DNF-AUTOMATIC with the new DNF5 and new Libdnf5 library. The change will mainly impact command-line users of DNF. It is a second step after https://fedoraproject.org/wiki/Changes/MajorUpgradeOfMicrodnf.

Owner

• Name: Jaroslav Mracek, Vendula Poncova
• Email: jmracek@redhat.com, vponcova@redhat.com

Current status

• Targeted release: Fedora Linux 39
• Last updated: 2022-12-07
• devel thread
• FESCo issue: #2870
• Tracker bug: <will be assigned by the Wrangler>
• Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

Currently the symlink /usr/bin/dnf points to /usr/bin/dnf-3 provided by python3-dnf. The change proposal is to

• Point the symlink /usr/bin/dnf to /usr/bin/dnf5 provided by dnf5.
• Ensure that DNF5 will be installed on the new system
• Provide upgrade path for systems with installed DNF to DNF5

Impacted users and components

• Commandline users and components (Significant)
  • Majority of user cases will be not affected
    • DNF symlink
    • Same syntaxes for common commands and options
  • Different outputs
  • Not all commands, options, or syntaxes will be supported
    • Dropping unused commands and options
    • Improve user experience by changing syntax
• DNF API users (Minor)
  • python3-dnf, libdnf remain in distribution and installed on the system => no issue
  • DNF5 and DNF will not share module state and history => it is not recommended to manage software on the system using DNF and DNF5 at the same time

Benefit to Fedora

The new DNF5 will provide a significant improvement in user experiences and performance. The replacement is the second step in upgrade of Fedora Software Management stack. Without the change there will be multiple software management tool (DNF5, old Microdnf, PackageKit, and DNF) based on different libraries (libdnf, libdnf5), providing a different behavior, and not sharing a history and module state. We can also expect that DNF will have only limited support from upstream. The DNF5 development was announced on Fedora-Devel list in 2020.

• Key features
  • Fully featured package manager without requirement of Python
  • DNF5 has smaller install size - 114 MB (Install size of DNF is 165 MB)
  • Significantly faster for many user cases
  • Reduction of number of software management tools in Fedora (DNF5 replaces DNF and Microdnf)
  • Optimization of download metadata
    • Optional download of filelists (configurable from config and commands)

• Unified behavior of in the software management stack
  • Same user experience in workstation, server, containers
    • DNF, YUM, MICRODNF commands will be linked to DNF5
  • New Libdnf5 plugins (C++, Python) will be applicable to DNF5, Dnf5Daemon
    • DNF4 plugins were not applicable for PackageKit and Microdnf (e.g. versionlock, subscription-manager), therefore PackageKit behaves differently in comparison to DNF
  • Shared configurations
    • In DNF4 not all configuration is honored by PackageKit and Microdnf
  • DNF/YUM was developed for decades with impact of multiple styles and naming conventions (options, configuration, options, commands)

• New Daemon
  • The new daemon can provide an alternative to PackageKit for RPMs (only one backend of PackageKit) if it will be integrated into Desktop
  • Support of Modularity and Comps group
• Performance improvement
  • Loading of repositories
  • Advisory operations
  • RPM queries
    • Name filters with a case-insensitive search (the repoquery command)
  • Smart sharing of metadata between dnf5 and daemon
    • Reduce disk and downloads requirements
    • Currently, DNF, Microdnf, and PackageKit use their own cache
    • Optional, may be not available for Fedora 39

• Decrease of a maintenance cost in the long term
  • Shared plugins
  • Removal of functional duplicates

• Fully integrated Modularity in LIBDNF5 workflows
  • The Modularity is supported in DNF and LIBDNF but it is not fully integrated. Integration was not possible due to limitation of compatibility with other tools (PackageKit)
  • Fully integrated Modularity required changes in the library workflow

Major codebase improvements

• Reports in structure
  • DNF reports a lot of important information only in logs

• Removal of duplicated implementation
  • LIBDNF evolved from LIBHIF (PackageKit library) and HAWKEY (DNF library). The integration was never finished, therefore LIBDNF still contains duplicated functionality.
  • decrease of the code maintenance cost in future

• Unify Python bindings
  • Formal Libdnf provides two types of Python bindings
    • CPython (hawkey)
    • SWIG (libdnf)
  • Maintaining and communication between both bindings requires a lot of resources
  • Binding unification was not possible without breaking API compatibility

• SWIG bindings
  • With SWIG we can generate additional bindings without spending huge resources
  • Code in particular languages will be very similar to each other

• Separation of system state from history DB and /etc/dnf/module.d
  • In dnf-4 the list of userinstalled packages and list of installed groups along with the lists of packages installed from them is computed as an aggregation of transaction history. In dnf5 it will be stored separately, having multiple benefits, among them that the history database will serve for informational purposes only and will not define the state of the system (it gets corrupted occasionally etc.).
  • Data stored in /etc/dnf/module.d were not supposed to be user modifiable and their format is not sufficient (missing information about installed packages with installed profiles)
    • Content of /etc/dnf/module.d will be moved into the System State

Problems related to using DNF5 and DNF in parallel for software modification

1. History in DNF and DNF5 are not shared

1.a. Information about installed dependencies are not propagated correctly

2. Modules

After implementation of dnf5 module install dnf and dnf5 cannot share the same location for storing state of modules, because dnf5 will store additional information about installed profiles (braking change).

Scope

• Proposal owners:

• Obsolete dnf package by dnf5
  • python3-dnf (provider of dnf-3 binary and dnf API) will be not affected by the change
• Modify comps groups to replace dnf or yum by dnf5
• DNF5 documentation [in progress]
  • https://dnf5.readthedocs.io/en/latest/dnf5.8.html
• DNF5 API documentation [enhancing]
  • https://dnf5.readthedocs.io/en/latest/api/index.html
• DNF5 API tutorials [DONE]
  • https://dnf5.readthedocs.io/en/latest/tutorial/index.html
• Testing repository [DONE]
  • https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-unstable/
• The scope of the features for Fedora 39
  • https://github.com/rpm-software-management/dnf5/milestone/1
  • Critical feature requested by community
    • Repoquery command with unknown scope
      • Tracking issue - https://github.com/rpm-software-management/dnf5/issues/122
    • Search command
      • Tracking issue - https://github.com/rpm-software-management/dnf5/issues/152
    • System-upgrade Command
    • DNF-automatic
  • Optional feature requested by community
    • Copr plugin
    • Snapper plugin
    • Config-manager
• The list of implemented commands and options
  • https://dnf5.readthedocs.io/en/latest/commands/index.html

The project's github repository is here - https://github.com/rpm-software-management/dnf5/

• Other developers:
  • anaconda
  • ansible
    • tracking issue - https://github.com/ansible/ansible/issues/78898
  • pungi
  • releng tools that call dnf repoquery

• Release engineering: #Releng issue number

• Policies and guidelines: N/A (not needed for this Change)

• Trademark approval: N/A (not needed for this Change)

• Alignment with Objectives:

Upgrade/compatibility impact

The new DNF5 will obsolete dnf, yum, and dnf-automatic. dnf and yum must be obsoleted because DNF5 will provide dnf and yum symlink. To ensure proper functionality dnf-automatic must be also obsoleted because DNF5 will store module state at the different location and format and DNF5 does not share history DB with DNF.

• Upgrade from Fedora 38
  • dnf package will be obsoleted by DNF5, dnf command will be redirected to DNF5 (DNF5 will provide a symlink to /usr/bin/dnf)
  • Original dnf-3 binary (provided by python3-dnf) will remain on the system - all functionality of dnf will be still accessible using dnf-3 binary.
  • If present yum package will be obsoleted by DNF5, yum command will be redirected to DNF5
  • dnf-automatic will be obsoleted by new tool based on LIBDNF5
    • Requirements for a modification of the configuration for the new tool is unknown - will be determine later

The current proposal is not about remove of current dnf packages from repostories. If someone wants or need to continue to use dnf, it would only require to exclude dnf5 in configuration prior upgrade to Fedora 39 (dnf, yum symlink will be still owned by original dnf and yum package).

The new DNF5 will provide a symlink to /usr/bin/dnf therefore users will see the replacement as an upgrade of DNF with limited but documented syntax changes. The DNF5 will provide some compatible aliases of commands and options to improve adoption of the DNF5. DNF5 will provide different outputs.

Compatibility and Stability

We adopt CI from DNF for DNF5 to ensure stability, discover differences and missing functionality.

How To Test

Install dnf5 package from https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-unstable/

User Experience

• Improved progress bars
• Improved transaction table
• Transaction progress reports including scriptlets reports
• Support of local rpm for transaction operation
• Great bash completion (better then DNF has)
• New commands and options that are only available with DNF

Dependencies

There is a long list of dependent packages

dnf

auter
calamares
copr-builder
cpanspec
dnf-plugin-diff
dnfdragora
etckeeper-dnf
fedora-review
fedora-upgrade
kiwi-systemdeps-core
libdnf-plugin-subscription-manager
lpf
mock
osbuild
perl-CPAN-Plugin-Sysdeps
policycoreutils-devel
rbm
subscription-manager
supermin
system-config-language

python3-dnf

anaconda-core
dnf-plugin-ovl
dnfdaemon
fedora-easy-karma
fedora-review
lorax
mock-core-configs
module-build-service
modulemd-tools
needrestart
pungi
python3-bodhi-client
python3-dnf-plugin-cow
python3-dnf-plugin-flunk_dependent_remove
python3-imgcreate
python3-libreport
retrace-server
system-config-language

libdnf

PackageKit
copr-builder
gnome-software-rpm-ostree
libdnf-plugin-subscription-manager
libdnf-plugin-swidtags
libdnf-plugin-txnupd

python3-hawkey

mock-core-configs
modulemd-tools
python3-rpmdeplint
retrace-server

Contingency Plan

• Contingency mechanism: Removal of dnf obsolete and dnf symlink from DNF5 build
• Contingency deadline: Mass rebuild
• Blocks release? No
  • If DNF5 will be not ready to replace DNF
• Original DNF component will be still available in Fedora repositories
  • If anyone will experience an unsolvable issue with DNF5
    • dnf-3 binary from python3-dnf can be used to temporary resolve the issue
      • DNF5 will document this approach including description of risks
    • DNF5 can be replaced by original DNF

Documentation

DNF5 documentation - https://dnf5.readthedocs.io/en/latest/