Latest revision as of 17:56, 18 July 2023

DNF Make Best Mode the Default

This is a rejected Change proposal for Fedora Linux.
This document represents a proposed Change that has been rejected or withdrawn as part of the Changes process. This page is preserved for historical record.

Summary

Currently, DNF prefers clean dependency resolution over package updates; a package (almost) silently won't get updated to a newer version if the new version has dependency problems. DNF will be changed to prefer updates and fail if they have dependency resolution issues, while the failure has a temporal or permanent workaround hint for users who want to use the older behavior.

Owner

Name: Jaroslav Mracek
Email: jmracek@redhat.com
Release notes owner:

Current status

Targeted release: Fedora 31
Last updated: 2023-07-18
FESCo issue: #2168
Tracker bug: <will be assigned by the Wrangler>

Detailed Description

Change the built-in default value of the best configuration option from 0 (false) to 1 (true).

As a result, unless best is overridden in the /etc/dnf/dnf.conf file or using --setopt, it will default to 1. As a convenience, we will also put the explicit best=1 assignment in the shipped /etc/dnf/dnf.conf file for better transparency, and introduce the new --nobest command-line switch.

The purpose of the --nobest switch (as a shorthand for --setopt=best=0) is to make it easy for the user to override the default setting when needed, and it will also be suggested in the DNF output when a dependency error occurs.

Relevant excerpt from the updated dnf.conf(5):

best  boolean
When upgrading a package, always try to install its highest version available, even only to find out some of its deps are not satisfiable. Enable this if you want to experience broken dependencies in the repositories firsthand. The default is True.

Relevant excerpt from the updated dnf(8):

--nobest
Set best option as false, therefore transactions are not limited to only best candidates.

Change in DNF output - missing vim-enhanced-2:8.1.1561-1.fc30

Original output. DNF succeed with return code 0:

sudo dnf upgrade
Last metadata expiration check: 2:16:40 ago on Mon 24 Jun 2019 04:27:16 PM CEST.
Dependencies resolved.

 Problem: package vim-enhanced-2:8.1.1471-1.fc30.x86_64 requires vim-common = 2:8.1.1471-1.fc30, but none of the providers can be installed
  - cannot install both vim-common-2:8.1.1561-1.fc30.x86_64 and vim-common-2:8.1.1471-1.fc30.x86_64
  - problem with installed package vim-enhanced-2:8.1.1471-1.fc30.x86_64
  - cannot install the best update candidate for package vim-common-2:8.1.1471-1.fc30.x86_64
  - package vim-enhanced-2:8.1.1561-1.fc30.x86_64 is excluded
===================================================================================================================================
 Package                        Architecture               Version                                Repository                   Size
===================================================================================================================================
Skipping packages with conflicts:
(add '--best --allowerasing' to command line to force their upgrade):
 vim-common                     x86_64                     2:8.1.1561-1.fc30                      updates                     6.7 M

Transaction Summary
===================================================================================================================================
Skip  1 Package

Nothing to do.
Complete!

Output after the change. DNF fails with return code 1, but proposing --nobest option as an option to resolve the issue:

sudo dnf upgrade
Last metadata expiration check: 2:16:36 ago on Mon 24 Jun 2019 04:27:16 PM CEST.
Error: 
 Problem: package vim-enhanced-2:8.1.1471-1.fc30.x86_64 requires vim-common = 2:8.1.1471-1.fc30, but none of the providers can be installed
  - cannot install both vim-common-2:8.1.1561-1.fc30.x86_64 and vim-common-2:8.1.1471-1.fc30.x86_64
  - problem with installed package vim-enhanced-2:8.1.1471-1.fc30.x86_64
  - cannot install the best update candidate for package vim-common-2:8.1.1471-1.fc30.x86_64
  - package vim-enhanced-2:8.1.1561-1.fc30.x86_64 is excluded
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Q&A

Can be a default of the best configuration option overwritten easily and permanently by user? Yes, just add best=false to /etc/dnf/dnf.conf

[main]
best=False

Can be a default of the best configuration option overwritten easily from commandline? Yes, just add --nobest to command

dnf upgrade --nobest

What about PackageKit and Gnome Software?

PackageKit and Gnome Software will be not affected by the change. In case that the same behavior will be desired for PackageKit, It will require changes in PackageKit code.

What about Microdnf?

Microdnf will be not affected by the change. There is a plan to unify functional parity and behavior DNF with Microdnf but not before Fedora 33.

Benefit to Fedora

This change allows the users to be properly notified when a package cannot be upgraded to the latest version, instead of silently ignoring it as an upgrade candidate.

Right now, when DNF runs in best=0 mode, if a package cannot be upgraded due to dependency problems, it is skipped and a warning is printed in the transaction summary table. However, this poses a risk of important security fixes being overlooked by the user in case they are broken for some reason, such as due to a repository misconfiguration or inconsistency within the metadata itself.

Moreover, since DNF always exits with the return code 0 (success) when in best=0 mode, this mode is especially risky in automated scripts invoking DNF in assumeyes mode in which case such unsuccessful package upgrades could easily go unnoticed unless the logs are manually examined after the fact.

The new behavior is also more in line with the generally accepted software development practice of failing early and failing fast.

As a secondary benefit, broken upgrade paths in the Fedora repositories will hopefully be noticed, reported and therefore fixed sooner. Although, we would prefer if such problems would be detected before we ship them to users.

Summary of benefits:

No silently passed problems with updates
Broken dependencies faster disappear from Fedora distribution
Problems will be reported more often - opportunity to fix issues
Increase in stability of Fedora distribution
Less issues after branching
Identical behavior of DNF in all distributions - Fedora/RHEL/Mageia/OpenSuse

Scope

Proposal owners:

The change is already part of the upstream (dnf-4.1.0) and reverted in Fedora downstream. The change was composed by following pull requests:

https://github.com/rpm-software-management/libdnf/pull/678
https://github.com/rpm-software-management/dnf/pull/1311
https://github.com/rpm-software-management/dnf/pull/1316
https://github.com/rpm-software-management/dnf/pull/1319

We would like to stop the reverting the changes.

Other developers: N/A (not a System Wide Change)

Release engineering: #Releng issue number (a check of an impact with Release Engineering is needed)
- List of deliverables: N/A (not a System Wide Change)

Policies and guidelines: N/A (not a System Wide Change)

Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

N/A (not a System Wide Change)

User Experience

Broken upgrades are recognized early, enabling the users to act upon them by double-checking their repository configuration or filing bugs, instead of assuming no upgrades are available.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

If there is a massive negative feedback by the rawhide and pre-beta users, we can revert the change at the beta freeze. If there is a massive negative feedback by the beta users, we can revert the change at final freeze.

Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
Contingency deadline: N/A (not a System Wide Change)
Blocks release? N/A (not a System Wide Change), Yes/No
Blocks product? product

Documentation

N/A (not a System Wide Change)

Release Notes

Search

Changes/DNF Default Best: Difference between revisions

Latest revision as of 17:56, 18 July 2023

Contents

DNF Make Best Mode the Default

Summary

Owner

Current status

Detailed Description

Benefit to Fedora

Scope

Upgrade/compatibility impact

How To Test

User Experience

Dependencies

Contingency Plan

Documentation

Release Notes

@@ Line 1: / Line 1: @@
-{{admon/important | Comments and Explanations | The page source contains comments providing guidance to fill out each section. They are invisible when viewing this page. To read it, choose the "view source" link.<br/> '''Copy the source to a ''new page'' before making changes!  DO NOT EDIT THIS TEMPLATE FOR YOUR CHANGE PROPOSAL.'''}}
-<!-- Self Contained or System Wide Change Proposal?
-Use this guide to determine to which category your proposed change belongs to.
-Self Contained Changes are:
-* changes to isolated/leaf package without the impact on other packages/rest of the distribution
-* limited scope changes without the impact on other packages/rest of the distribution
-* coordinated effort within SIG with limited impact outside SIG functional area, accepted by the SIG
-System Wide Changes are:
-* changes that does not fit Self Contained Changes category touching
-* changes that require coordination within the distribution (for example mass rebuilds, release engineering or other teams effort etc.)
-* changing system defaults
-For Self Contained Changes, sections marked as "REQUIRED FOR SYSTEM WIDE CHANGES" are OPTIONAL but FESCo/Wrangler can request more details (especially in case the change proposal category is
-improper or updated to System Wide category). For System Wide Changes all fields on this form are required for FESCo acceptance (when applies).
-We request that you maintain the same order of sections so that all of the change proposal pages are uniform.
--->
 <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
+= DNF Make Best Mode the Default <!-- The name of your change proposal --> =
-<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
+{{Change_Rejected_Banner}}
-= DNF Best Mode By Default <!-- The name of your change proposal --> =
 == Summary ==
-Make DNF always try to upgrade to the highest version of a package available, even only to fail due to dependency issues.
+Currently, DNF prefers clean dependency resolution over package updates;
-<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release.
+a package (almost) silently won't get updated to a newer version if the new
-Note that motivation for the change should be in the Motivation section below, and this part should answer the question "What?" rather than "Why?". -->
+version has dependency problems. DNF will be changed to prefer updates and fail
+if they have dependency resolution issues, while the failure has a temporal or permanent workaround
+hint for users who want to use the older behavior.
 == Owner ==
@@ Line 35: / Line 16: @@
 This should link to your home wiki page so we know who you are.
 -->
-* Name: [[User:mdomonko| Michal Domonkos]]
+* Name: [[User:jmracek| Jaroslav Mracek]]
 <!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
-* Email: mdomonko@redhat.com <your email address so we can contact you, invite you to meetings, etc.>
+* Email: jmracek@redhat.com
 * Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 <!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
@@ Line 48: / Line 29: @@
 == Current status ==
-* Targeted release: [[Releases/30 | Fedora 30 ]]
+* Targeted release: [[Releases/31 | Fedora 31 ]]
 * Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}
+* FESCo issue: [https://pagure.io/fesco/issue/2168 #2168]
 <!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page
 Bugzilla states meaning as usual:
@@ Line 61: / Line 43: @@
 == Detailed Description ==
+Change the built-in default value of the `best` configuration option from `0` (false) to `1` (true).
+As a result, unless `best` is overridden in the `/etc/dnf/dnf.conf` file or using `--setopt`, it will default to `1`. As a convenience, we will also put the explicit `best=1` assignment in the shipped `/etc/dnf/dnf.conf` file for better transparency, and introduce the new `--nobest` command-line switch.
+The purpose of the `--nobest` switch (as a shorthand for `--setopt=best=0`) is to make it easy for the user to override the default setting when needed, and it will also be [https://github.com/rpm-software-management/dnf/pull/1311/commits/9a3e8fd0da49291d30fd1fef527cffb0bf3f047d#diff-6c823931c6d150295e5011fac6529ab9R144 suggested] in the DNF output when a dependency error occurs.
+Relevant excerpt from the updated `dnf.conf(5)`:
+<pre>
+best  boolean
+When upgrading a package, always try to install its highest version available, even only to find out some of its deps are not satisfiable. Enable this if you want to experience broken dependencies in the repositories firsthand. The default is True.
+</pre>
+Relevant excerpt from the updated `dnf(8)`:
+<pre>
+--nobest
+Set best option as false, therefore transactions are not limited to only best candidates.
+</pre>
+'''Change in DNF output - missing vim-enhanced-2:8.1.1561-1.fc30'''
+Original output. DNF succeed with return code 0:
+<pre>
+sudo dnf upgrade
+Last metadata expiration check: 2:16:40 ago on Mon 24 Jun 2019 04:27:16 PM CEST.
+Dependencies resolved.
+ Problem: package vim-enhanced-2:8.1.1471-1.fc30.x86_64 requires vim-common = 2:8.1.1471-1.fc30, but none of the providers can be installed
+  - cannot install both vim-common-2:8.1.1561-1.fc30.x86_64 and vim-common-2:8.1.1471-1.fc30.x86_64
+  - problem with installed package vim-enhanced-2:8.1.1471-1.fc30.x86_64
+  - cannot install the best update candidate for package vim-common-2:8.1.1471-1.fc30.x86_64
+  - package vim-enhanced-2:8.1.1561-1.fc30.x86_64 is excluded
+===================================================================================================================================
+ Package                        Architecture               Version                                Repository                   Size
+===================================================================================================================================
+Skipping packages with conflicts:
+(add '--best --allowerasing' to command line to force their upgrade):
+ vim-common                     x86_64                     2:8.1.1561-1.fc30                      updates                     6.7 M
+Transaction Summary
+===================================================================================================================================
+Skip  1 Package
+Nothing to do.
+Complete!
+</pre>
+Output after the change. DNF fails with return code 1, but proposing `--nobest` option as an option to resolve the issue:
+<pre>
+sudo dnf upgrade
+Last metadata expiration check: 2:16:36 ago on Mon 24 Jun 2019 04:27:16 PM CEST.
+Error:
+ Problem: package vim-enhanced-2:8.1.1471-1.fc30.x86_64 requires vim-common = 2:8.1.1471-1.fc30, but none of the providers can be installed
+  - cannot install both vim-common-2:8.1.1561-1.fc30.x86_64 and vim-common-2:8.1.1471-1.fc30.x86_64
+  - problem with installed package vim-enhanced-2:8.1.1471-1.fc30.x86_64
+  - cannot install the best update candidate for package vim-common-2:8.1.1471-1.fc30.x86_64
+  - package vim-enhanced-2:8.1.1561-1.fc30.x86_64 is excluded
+(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
+</pre>
+'''Q&A'''
+Can be a default of the best configuration option overwritten easily and permanently by user?
+Yes, just add `best=false` to `/etc/dnf/dnf.conf`
+<pre>
+[main]
+best=False
+</pre>
+Can be a default of the best configuration option overwritten easily from commandline?
+Yes, just add `--nobest` to command
+<pre>
+dnf upgrade --nobest
+</pre>
+What about PackageKit and Gnome Software?
+<pre>
+PackageKit and Gnome Software will be not affected by the change. In case that the same behavior will be desired for PackageKit, It will require changes in PackageKit code.
+</pre>
+What about Microdnf?
+<pre>
+Microdnf will be not affected by the change. There is a plan to unify functional parity and behavior DNF with Microdnf but not before Fedora 33.
+</pre>
-<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
 == Benefit to Fedora ==
-<!-- What is the benefit to the distribution?  Will the software we generate be improved? How will the process of creating Fedora releases be improved?
+This change allows the users to be properly notified when a package cannot be upgraded to the latest version, instead of silently ignoring it as an upgrade candidate.
-      Be sure to include the following areas if relevant:
+Right now, when DNF runs in `best=0` mode, if a package cannot be upgraded due to dependency problems, it is skipped and a warning is printed in the transaction summary table. However, this poses a risk of important security fixes being overlooked by the user in case they are broken for some reason, such as due to a repository misconfiguration or inconsistency within the metadata itself.
-      If this is a major capability update, what has changed?
-           For example: This change introduces Python 5 that runs without the Global Interpreter Lock and is fully multithreaded.
+Moreover, since DNF always exits with the return code `0` (success) when in `best=0` mode, this mode is especially risky in automated scripts invoking DNF in `assumeyes` mode in which case such unsuccessful package upgrades could easily go unnoticed unless the logs are manually examined after the fact.
-      If this is a new functionality, what capabilities does it bring?
-           For example: This change allows package upgrades to be performed automatically and rolled-back at will.
+The new behavior is also more in line with the generally accepted software development practice of failing early and failing fast.
-      Does this improve some specific package or set of packages?
-           For example: This change modifies a package to use a different language stack that reduces install size by removing dependencies.
+As a secondary benefit, broken upgrade paths in the Fedora repositories will hopefully be noticed, reported and therefore fixed sooner. Although, we would prefer if such problems would be detected before we ship them to users.
-      Does this improve specific Spins or Editions?
-           For example: This change modifies the default install of Fedora Workstation to be more in line with the base install of Fedora Server.
-      Does this make the distribution more efficient?
-           For example: This change replaces thousands of individual %post scriptlets in packages with one script that runs at the end.
-      Is this an improvement to maintainer processes?
-           For example: Gating Fedora packages on automatic QA tests will make rawhide more stable and allow changes to be implemented more smoothly.
-      Is this an improvement targeted as specific contributors?
-           For example: Ensuring that a minimal set of tools required for contribution to Fedora are installed by default eases the onboarding of new contributors.
-     When a Change has multiple benefits, it's better to list them all.
+'''Summary of benefits:'''
-     Consider these Change pages from previous editions as inspiration:
+# No silently passed problems with updates
-     https://fedoraproject.org/wiki/Changes/Annobin (low-level and technical, invisible to users)
+# Broken dependencies faster disappear from Fedora distribution
-     https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo (low-level, but visible to advanced users)
+# Problems will be reported more often - opportunity to fix issues
-     https://fedoraproject.org/wiki/Changes/VirtualBox_Guest_Integration (primarily a UX change)
+# Increase in stability of Fedora distribution
-     https://fedoraproject.org/wiki/Changes/NoMoreAlpha (an improvement to distro processes)
+# Less issues after branching
-     https://fedoraproject.org/wiki/Changes/perl5.26 (major upgrade to a popular software stack, visible to users of that stack)
+# Identical behavior of DNF in all distributions - Fedora/RHEL/Mageia/OpenSuse
--->
 == Scope ==
 * Proposal owners:
-<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
+The change is already part of the upstream (dnf-4.1.0) and reverted in Fedora downstream. The change was composed by following pull requests:
+https://github.com/rpm-software-management/libdnf/pull/678<br>
+https://github.com/rpm-software-management/dnf/pull/1311<br>
+https://github.com/rpm-software-management/dnf/pull/1316<br>
+https://github.com/rpm-software-management/dnf/pull/1319
+We would like to stop the reverting the changes.
 * Other developers: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
@@ Line 140: / Line 200: @@
 == User Experience ==
-<!-- If this change proposal is noticeable by users, how will their experiences change as a result?
+Broken upgrades are recognized early, enabling the users to act upon them by double-checking their repository configuration or filing bugs, instead of assuming no upgrades are available.
- This section partially overlaps with the Benefit to Fedora section above. This section should be primarily about the User Experience, written in a way that does not assume deep technical knowledge. More detailed technical description should be left for the Benefit to Fedora section.
- Describe what Users will see or notice, for example:
-  - Packages are compressed more efficiently, making downloads and upgrades faster by 10%.
-  - Kerberos tickets can be renewed automatically. Users will now have to authenticate less and become more productive. Credential management improvements mean a user can start their work day with a single sign on and not have to pause for reauthentication during their entire day.
- - Libreoffice is one of the most commonly installed applications on Fedora and it is now available by default to help users "hit the ground running".
- - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
--->
 == Dependencies ==
@@ Line 158: / Line 209: @@
 == Contingency Plan ==
+If there is a massive negative feedback by the rawhide and pre-beta users, we can revert the
+change at the beta freeze. If there is a massive negative feedback by the beta users, we can
+revert the change at final freeze.
 <!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->