Changes/DisableSTI

Comments and Explanations
The page source contains comments providing guidance to fill out each section. They are invisible when viewing this page. To read it, choose the "view source" link.
Copy the source to a new page before making changes! DO NOT EDIT THIS TEMPLATE FOR YOUR CHANGE PROPOSAL.

Guidance
For details on how to fill out this form, see the documentation.

Report issues
To report an issue with this template, file an issue in the pgm_docs repo.

Deprecation of STI tests

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Display a deprecation warning for Fedora 41 for all STI tests. Deprecate execution of STI tests in all CI pipelines for Fedora 42.

• CI for bodhi updates
• CI for dist-git pull requests

All users of STI tests will need to migrate to the new tmt format.

The change will affect 281 components. The list of components affected can be found via this sourcegraph query.

All references to STI will be removed from the Fedora CI documentation.

Owner

• Name: Miroslav Vadkerti, Cristian Le,
• Email: mvadkert@redhat.com, fedora@lecris.me

Current status

• Targeted release: Fedora Linux 41
• Last updated: 2025-01-03
• [Announced]
• [<will be assigned by the Wrangler> Discussion thread]
• FESCo issue: <will be assigned by the Wrangler>
• Tracker bug: <will be assigned by the Wrangler>
• Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

For some time CI testing in Fedora can be defined using two different formats:

• Standard Test Interface (STI)
• Test Management Tool (tmt)

As tmt has matured to a state it can fully replace the STI functionality, we are proposing to drop STI as the supported format. STI format is not actively developed and is causing us a maintenance burden. More and more STI test failures are starting to appear as the Fedora packages evolve out of sync with the test scripts.

Tmt provides various advantages over STI that would make it easier to manage in the long term:

• Better organization of tests and test environments (referred to as plans in tmt)
• Local reproducible environment thanks to testing-farm reproducer script, allowing to thinker locally without pushing to dist-git
• Tests can be defined in the local dist-git repo, inside the srpm archive, tests/* dist-git namespace, or in the upstream repo, allowing it to be reused more freely between packages and with upstream directly
• Tmt tests are integrated with packit allowing it to be executed with upstream and better migrate to the new dist-git environment

Hopefully this list can help inspire some better ways of reorganizing the tests during the migration.

Benefit to Fedora

Having two formats for executing the tests is an unnecessary duplication and causes confusion for the Fedora maintainers and community.

STI tests have limited functionality and are harder to develop and maintain, when compared to tmt.

Scope

• Proposal owners: Miroslav Vadkerti, Cristian Le

• Other developers:

• Release engineering: #Releng issue number

• Policies and guidelines: N/A (not needed for this Change)

• Trademark approval: N/A (not needed for this Change)

• Alignment with the Fedora Strategy:

Upgrade/compatibility impact

This is a change to the development experience, no changes to Fedora distribution are made.

How To Migrate

To find out if you package has STI tests, check if tests/tests*.yml files are present in your dist-git repository.

Follow the sti to tmt migration guide. For the majority of tests involve just executing the already provided script you only need to:

1. Initialize tmt structure using tmt init

2. Create a minimal plan

# plan.fmf
# Boilerplate to indicate tmt to run any tests files it finds in the repo
discover:
  how: fmf
execute:
  how: tmt
# Prepare test environment, e.g. install packages
# Note: the current packages in the spec file will already be installed by default
prepare:
  how: install
  packages:
    - foo
    - bar

3. Migrate individual tests

# tests/foo.fmf
test: ./foo.sh

For more complex needs feel free to discuss in the #fedora-ci or #tmt matrix channels. You can also explore other project's tmt tests (identified by having a .fmf folder) using tmt tests show and tmt plans show

User Experience

For Fedora 41 users will be presented with a deprecation warning in the Testing Farm artifacts.

For Fedora 42 users who have setup gating on STI tests will be obligated to migrate to be able to push their packages to stable repositories.

Dependencies

N/A

Contingency Plan

• Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
• Contingency deadline: N/A (not a System Wide Change)
• Blocks release? No (not a System Wide Change)

We will keep the STI support until all references to STI tests are gone in case all users have not migrated to tmt until Fedora 42. See the description for the list of the affected components.

Documentation

• How to migrate STI tests to tmt
• Example test repo implemented in both STI and tmt format (see for example the smoke/venv.fmf file and smoke_default test in tests.yml file)
• Tmt specs (reference for the available keys defining plans or tests)
• Example project with shared upstream+downstream tests