From Fedora Project Wiki

< Changes
Revision as of 13:00, 17 January 2025 by Mvadkert (talk | contribs)

Deprecation of STI tests

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Drop suport for Standard Test Interface (STI) tests for Fedora 42 and later.

For Fedora 42:

  • stop running STI tests after the branch date February 4, 2025 for the Bodhi updates.
  • keep running STI tests for Fedora 42 dist-git pull requests until the release date.

For EPEL:

  • keep running STI tests for EPEL dist-git pull requests until Fedora 42 release date.

Keep running the STI tests for Fedora 40 and Fedora 41.

All users of STI tests will need to migrate to the new tmt format during the Fedora 42 development.

The change will affect 281 components. The list of components affected can be found via this sourcegraph query.

All references to STI will be removed from the Fedora CI documentation.

Owner



Current status

  • Targeted release: Fedora Linux 42
  • Last updated: 2025-01-17
  • Announced
  • Discussion thread
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

For some time CI testing in Fedora can be defined using two different formats:

  • Standard Test Interface (STI)
  • Test Management Tool (tmt)

As tmt has matured to a state it can fully replace the STI functionality, we are proposing to drop STI as the supported format. STI format is not actively developed and is causing us a maintenance burden. More and more STI test failures are starting to appear as the Fedora packages evolve out of sync with the test scripts.

Tmt provides various advantages over STI that would make it easier to manage in the long term:

  • Better organization of tests and test environments (referred to as plans in tmt)
  • Local reproducible environment thanks to testing-farm reproducer script, allowing to thinker locally without pushing to dist-git
  • Tests can be defined in the local dist-git repo, inside the srpm archive, tests/* dist-git namespace, or in the upstream repo, allowing it to be reused more freely between packages and with upstream directly
  • Tmt tests are integrated with packit allowing it to be executed with upstream and better migrate to the new dist-git environment

Hopefully this list can help inspire some better ways of reorganizing the tests during the migration.

Feedback

Benefit to Fedora

Having two formats for executing the tests is an unnecessary duplication and causes confusion for the Fedora maintainers and community.

STI tests have limited functionality and are harder to develop and maintain, when compared to tmt.

Scope


  • Other developers:
  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with the Fedora Strategy:

Upgrade/compatibility impact

This is a change to the development experience, no changes to Fedora distribution are made.

How To Migrate

To find out if you package has STI tests, check if tests/tests*.yml files are present in your dist-git repository.

Follow the sti to tmt migration guide. For the majority of tests involve just executing the already provided script you only need to:

1. Initialize tmt structure using tmt init

2. Create a minimal plan 

# /plan.fmf
# Boilerplate to indicate tmt to run any tests files it finds in the repo
discover:
  how: fmf
execute:
  how: tmt
# Prepare test environment, e.g. install packages
# Note: the current packages in the spec file will already be installed by default
prepare:
  how: install
  packages:
    - foo
    - bar

3. Migrate individual tests 

# /tests/foo.fmf
test: ./foo.sh

For more complex needs feel free to discuss in the #fedora-ci or #tmt matrix channels. You can also explore other project's tmt tests (identified by having a .fmf folder) using tmt tests show and tmt plans show

User Experience

For Fedora 41 users will be presented with a deprecation warning in the Testing Farm artifacts.

For Fedora 42 users who have setup gating on STI tests will be obligated to migrate to be able to push their packages to stable repositories.

Dependencies

N/A

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No (not a System Wide Change)

We will keep the STI support until all references to STI tests are gone in case all users have not migrated to tmt until Fedora 42. See the description for the list of the affected components.

Documentation

Release Notes

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/DisableSTI&oldid=728821"