Changes/Module Obsoletes and EOL

Introduce module Obsoletes and EOL

Summary

Fix Fedora upgradability issues when upgrading systems with module streams enabled.

Owner

• Name: Daniel Mach
• Email: dmach@redhat.com

Current status

• Targeted release: Fedora 33
• Last updated: 2020-05-12
• FESCo issue: #2364
• Tracker bug: #1834844
• Release notes tracker: #506

Detailed Description

DNF currently doesn't have sufficient information to perform system upgrades of systems with enabled module streams correctly. To solve upgradability problems, we will add additional modulemd metadata. This includes information about a stream obsoleting another one or a stream being EOLed.

Use Cases:

• A module (all its streams) is removed from a distribution.
• A stream has reached its EOL and must no longer be available.
  • The stream packages have no replacements and must be removed from the system (an extension of fedora-obsolete-packages?)
  • The stream packages get replaced with non-modular packages
• A stream has reached its EOL and is Obsoleted by another stream.
• A stream has reached its EOL and a user wants to continue using it regardless of EOL/Obsoletes.
• A stream has reached its EOL and a user wants to install a new host with the EOLed stream enabled.

There is going to be a new DNF config option:

• (name: TBD) - automatically follow stream EOL/Obsoletes; only print a warning if turned off

Proposed dnf behavior:

• dnf system-upgrade will automatically follow module Obsoletes and EOL, resetting or switching streams accordingly
• other dnf commands will not follow Obsoletes and EOL automatically; user will have to manually enable an option (see the paragraph above); this should help Rawhide users to handle module upgradability problems because it's a rolling release and they do not run system upgrade.

Proposed new modulemd document:

document: TBD
version: 1
data:
  # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ
  # When merging, entries with a newer 'modified' value will override any earlier values.
  # MANDATORY
  modified: 2020-03-19T23:26Z

  # A boolean option to cancel/reset all previously specified obsoletes
  # Example: repo 'fedora' has Obsoletes:nodejs:12; we want to bring nodejs:12 back in 'updates'
  # If used, following options will be ignored: eol_date, obsoleted_by
  # OPTIONAL
  <TBD>: <bool>

  # A string representing a Name of a module that is EOLed
  # MANDATORY
  module: nodejs

  # A string representing a Stream of a module that is EOLed
  # MANDATORY
  stream: 11

  # A string representing a Context of a module that is EOLed
  # If not specified, all contexts get EOLed.
  # NOTE: consider specifying a list of contexts
  # OPTIONAL
  context: aabbccddee

  # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ
  # It is strongly recommended to keep HH:MM to 00:00.
  # If not specified, the module is EOLed immediately.
  OPTIONAL
  eol_date: 2020-03-19T00:00Z

  # A string describing the change, reason, etc.
  # MANDATORY
  message: "Module stream nodejs:11 is no longer supported. It is recommended to switch to nodejs:12"

  # If a stream is not EOLed but Obsoleted, provide details about the obsoleting stream:
  # OPTIONAL
  obsoleted_by:
    module: nodejs
    stream: 12

Benefit to Fedora

Seamless system upgrades of systems with module streams enabled.

Scope

• Proposal owners:
  • Introduce modularity features to:
  • Obsolete a module stream
  • Stream EOL
  • Design a new modulemd documents for Obsoletes and EOL
  • Get the new documents supported by libmodulemd
  • Implement the new functionality in the DNF stack (libdnf, dnf, microdnf) and PackageKit

• Other developers:
  • Follow updated packaging policy. See the "Policies and guidelines" section.

• Release engineering: [1] (a check of an impact with Release Engineering is needed)

Maintain and distribute new module metadata.

• Policies and guidelines:
  • Packaging policy and modularity documents require a change:
    • When a module stream gets removed from a Fedora release, the maintainer of the module stream must provide a modulemd document with Obsoletes or EOL data.

• Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

This feature solves upgradability problems of systems with module streams enabled.

How To Test

Stream Obsoletes:

• Enable a stream that is not available in the next version of Fedora and is obsoleted by another stream in the metadata
• Run system upgrade
• Verify that the obsoleting stream is enabled instead of the obsoleted one

Stream EOL:

• Enable a stream that is not available in the next version of Fedora and is obsoleted in its module metadata
• Run system upgrade
• Verify that the EOLed stream is no longer enabled and does not show up in dnf module list

Fedora upgradability:

• Install Fedora 31 or 32
• For each $stream in available streams
  • dnf module reset '*'
  • dnf module enable $stream
  • dnf distro-sync --releasever=33 --assumeno
  • # the command must not throw any errors related to modularity, especially related to the enabled stream

User Experience

Fedora users experience upgradability problems when upgrading to Fedora 32 when they have any module streams enabled.

If a stream no longer exists on the version of Fedora they are upgrading to, DNF used to error out on resolving modular dependencies which was a clear release blocker. To workaround this case, all module streams are reset during the system upgrade. By doing this, modularity users lose information about enabled streams and they need to re-enable them by hand.

This Change aims at removing the upgradability problems and allowing Fedora modularity users to upgrade their systems while keeping their streams enabled, reset or switched (obsoleted) according to newly provided modular metadata.

Dependencies

• libmodulemd
• libdnf
• dnf
• microdnf
• PackageKit

Contingency Plan

• Contingency mechanism: Remove the new metadata from Fedora composes and repositories. Fedora will stay at the current behavior.
• Contingency deadline: Beta Freeze
• Blocks release? Yes

Documentation

TBD

Release Notes

Modularity supports Obsoleting streams and stream EOL (end-of-life). Enabled streams are no longer reset during the system upgrade. When a stream reaches its EOL, it's reset. When a stream is Obsoleted, it's switched to the obsoleting stream.