CI/Examples

On this page you can find some inspiration from real-life examples of tests already enabled in the Fedora CI.

Python

There are multiple versions of Python programming language available in Fedora and a number of related subpackages. As all of them should be tested (including their various combinatios) we share test coverage for them in the tests namespace:

• https://src.fedoraproject.org/tests/python

The test repo contains basic smoke test for virtualenv together with example test Metadata stored in the Flexible Metadata Format:

• main.fmf
• venv.fmf

Once the test is avaible in the share test repository it can be easily linked from supported Python versions:

• python2 python26
• python3 python34 python35 python36

We test additional Python implementations as well:

• pypy pypy3
• jython

Plus we ensure that essential tools for venv and virtualnv, such as setuptools, pip or virtualenv itself correctly work with all supported versions:

• python-pip
• python-wheel
• python-setuptools
• python-virtualenv
• python-tox

Note that for the last set of examples we run the same test several times with modified environment. For example:

   - smoke36:
       dir: python/smoke
       run: VERSION=3.6 ./venv.sh
   - smoke37:
       dir: python/smoke
       run: VERSION=3.7 ./venv.sh
   - smoke26:
       dir: python/smoke
       run: VERSION=2.6 METHOD=virtualenv TOX=false ./venv.sh
   - smoke27:
       dir: python/smoke
       run: VERSION=2.7 METHOD=virtualenv ./venv.sh
   - smoke34_virtualenv:
       dir: python/smoke
       run: VERSION=3.4 METHOD=virtualenv ./venv.sh

In this way we create several virtual test cases from a single test code which prevents duplication and minimizes future maintenance.

Shell

There are several shells which implement the POSIX specification: bash, ksh, mksh, zsh, dash. All of them share a significant amount of test coverage and it does not make sense to commit & maintain identical tests in five different repositories (+ possible branches). Thus we store test code in the tests namespace:

• https://src.fedoraproject.org/tests/shell

These tests are then linked from all relevant tests.yml files:

• bash
• ksh
• mksh
• zsh
• dash

Flexible Metadata Format filter is used to select appropriate tests instead of listing individual tests manually. Environment variables PACKAGES and SH_BIN are used to specify which shell implementation is being tested:

   - hosts: localhost
     roles:
     - role: standard-test-beakerlib
       tags:
       - classic
       repositories:
       - repo: "https://src.fedoraproject.org/tests/shell.git"
         dest: "shell"
         fmf_filter: "tier: 1, 2 & tags: classic"
       environment:
         PACKAGES: ksh
         SH_BIN: ksh
       required_packages:
       - ksh
       - expect            # login requires expect
       - which             # smoke requires which

Some of the tests might be relevant only for selected components. This can be handled easily by additional component condition:

   repositories:
   - repo: "https://src.fedoraproject.org/tests/shell.git"
     dest: "shell"
     fmf_filter: "tier: 1, 2 & component: dash"

See the Metadata page for the full list of so far drafted attributes.