|
|
| Line 1: |
Line 1: |
| On this page you can find some inspiration from real-life examples of tests already enabled in the Fedora CI.
| | Moved to: https://docs.fedoraproject.org/en-US/ci/examples/ |
| | |
| = did =
| |
| | |
| For each component it makes sense to enable even the most simple test such as running the binary with `--help` or using an internal smoke test. Here's an example from the [https://src.fedoraproject.org/rpms/did/pull-request/5 did] component:
| |
| | |
| - hosts: localhost
| |
| roles:
| |
| - role: standard-test-basic
| |
| tags:
| |
| - classic
| |
| tests:
| |
| - smoke:
| |
| dir: .
| |
| run: did this year --test
| |
| required_packages:
| |
| - did
| |
| | |
| That's it. As you see above, executing a single command as a test is very easy with the help of the [https://fedoraproject.org/wiki/CI/Standard_Test_Roles#Basic Basic] role.
| |
| | |
| = Python =
| |
| | |
| There are multiple versions of Python programming language available in Fedora and a number of related subpackages. As all of them should be tested (including their various combinatios) we [https://fedoraproject.org/wiki/CI/Share_Test_Code share] test coverage for them in the `tests` namespace:
| |
| | |
| * https://src.fedoraproject.org/tests/python
| |
| | |
| The test repo contains basic smoke test for virtualenv together with example test [https://fedoraproject.org/wiki/CI/Metadata Metadata] stored in the [https://fedoraproject.org/wiki/Flexible_Metadata_Format Flexible Metadata Format]:
| |
| | |
| * [https://src.fedoraproject.org/tests/python/blob/master/f/main.fmf main.fmf]
| |
| * [https://src.fedoraproject.org/tests/python/blob/master/f/smoke/venv.fmf venv.fmf]
| |
| | |
| Once the test is avaible in the share test repository it can be easily linked from supported Python versions:
| |
| | |
| * [https://src.fedoraproject.org/rpms/python2/blob/master/f/tests/tests.yml python2] [https://src.fedoraproject.org/rpms/python26/blob/master/f/tests/tests.yml python26]
| |
| * [https://src.fedoraproject.org/rpms/python3/blob/master/f/tests/tests.yml python3] [https://src.fedoraproject.org/rpms/python34/blob/master/f/tests/tests.yml python34] [https://src.fedoraproject.org/rpms/python35/blob/master/f/tests/tests.yml python35] [https://src.fedoraproject.org/rpms/python36/blob/master/f/tests/tests.yml python36]
| |
| | |
| We test additional Python implementations as well:
| |
| | |
| * [https://src.fedoraproject.org/rpms/pypy/blob/master/f/tests/tests.yml pypy]
| |
| * [https://src.fedoraproject.org/rpms/pypy3/blob/master/f/tests/tests.yml pypy3]
| |
| | |
| Plus we ensure that essential tools for venv and virtualnv, such as `setuptools`, `pip` or `virtualenv` itself correctly work with all supported versions:
| |
| | |
| * [https://src.fedoraproject.org/rpms/python-pip/blob/master/f/tests/tests.yml python-pip]
| |
| * [https://src.fedoraproject.org/rpms/python-wheel/blob/master/f/tests/tests.yml python-wheel]
| |
| * [https://src.fedoraproject.org/rpms/python-setuptools/blob/master/f/tests/tests.yml python-setuptools]
| |
| * [https://src.fedoraproject.org/rpms/python-virtualenv/blob/master/f/tests/tests.yml python-virtualenv]
| |
| * [https://src.fedoraproject.org/rpms/python-tox/blob/master/f/tests/tests.yml python-tox]
| |
| | |
| Note that for the last set of examples we run the same test several times with modified environment. For example:
| |
| | |
| - smoke36:
| |
| dir: python/smoke
| |
| run: VERSION=3.6 ./venv.sh
| |
| - smoke37:
| |
| dir: python/smoke
| |
| run: VERSION=3.7 ./venv.sh
| |
| - smoke26:
| |
| dir: python/smoke
| |
| run: VERSION=2.6 METHOD=virtualenv TOX=false ./venv.sh
| |
| - smoke27:
| |
| dir: python/smoke
| |
| run: VERSION=2.7 METHOD=virtualenv ./venv.sh
| |
| - smoke34_virtualenv:
| |
| dir: python/smoke
| |
| run: VERSION=3.4 METHOD=virtualenv ./venv.sh
| |
| | |
| In this way we create several virtual test cases from a single test code which prevents duplication and minimizes future maintenance.
| |
| | |
| = Shell =
| |
| | |
| There are several shells which implement the POSIX specification: bash, ksh, mksh, zsh, dash. All of them share a significant amount of test coverage and it does not make sense to commit & maintain identical tests in five different repositories (+ possible branches). Thus we store test code in the `tests` namespace:
| |
| | |
| * https://src.fedoraproject.org/tests/shell
| |
| | |
| These tests are then linked from all relevant `tests.yml` files:
| |
| | |
| * [https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml bash]
| |
| * [https://src.fedoraproject.org/rpms/ksh/blob/master/f/tests/tests.yml ksh]
| |
| * [https://src.fedoraproject.org/rpms/mksh/blob/master/f/tests/tests.yml mksh]
| |
| * [https://src.fedoraproject.org/rpms/zsh/blob/master/f/tests/tests.yml zsh]
| |
| * [https://src.fedoraproject.org/rpms/dash/blob/master/f/tests/tests.yml dash]
| |
| | |
| [https://fedoraproject.org/wiki/Flexible_Metadata_Format Flexible Metadata Format] filter is used to select appropriate tests instead of listing individual tests manually. Environment variables `PACKAGES` and `SH_BIN` are used to specify which shell implementation is being tested:
| |
| | |
| - hosts: localhost
| |
| roles:
| |
| - role: standard-test-beakerlib
| |
| tags:
| |
| - classic
| |
| repositories:
| |
| - repo: "https://src.fedoraproject.org/tests/shell.git"
| |
| dest: "shell"
| |
| fmf_filter: "tier: 1, 2 & tags: classic"
| |
| environment:
| |
| PACKAGES: ksh
| |
| SH_BIN: ksh
| |
| required_packages:
| |
| - ksh
| |
| - expect # login requires expect
| |
| - which # smoke requires which
| |
| | |
| Some of the tests might be relevant only for selected components. This can be handled easily by additional `component` condition:
| |
| | |
| repositories:
| |
| - repo: "https://src.fedoraproject.org/tests/shell.git"
| |
| dest: "shell"
| |
| fmf_filter: "tier: 1, 2 & component: dash"
| |
| | |
| See the [https://fedoraproject.org/wiki/CI/Metadata Metadata] page for the full list of so far drafted attributes.
| |
| | |
| = SELinux =
| |
| | |
| There are several components related to SELinux. They are tightly connected so change in one of them can cause problems in other. That's why their tests are shared and executed together:
| |
| | |
| * https://src.fedoraproject.org/tests/selinux
| |
| | |
| Instead of listing relevant tests to be executed manually in each dist git rpms repository [https://fedoraproject.org/wiki/Flexible_Metadata_Format Flexible Metadata Format] is used:
| |
| | |
| - hosts: localhost
| |
| roles:
| |
| - role: standard-test-beakerlib
| |
| tags:
| |
| - classic
| |
| repositories:
| |
| - repo: "https://src.fedoraproject.org/tests/selinux.git"
| |
| dest: "selinux"
| |
| fmf_filter: "tier: 1 | component: selinux-policy"
| |
| | |
| Provided `fmf_filter` selects all tests relevant for the `selinux-policy` component plus all Tier 1 selinux tests:
| |
| | |
| tier: 1 | component: selinux-policy
| |
| | |
| The following six components are covered:
| |
| | |
| * [https://src.fedoraproject.org/rpms/checkpolicy/blob/master/f/tests/tests.yml checkpolicy]
| |
| * [https://src.fedoraproject.org/rpms/libselinux/blob/master/f/tests/tests.yml libselinux]
| |
| * [https://src.fedoraproject.org/rpms/libsemanage/blob/master/f/tests/tests.yml libsemanage]
| |
| * [https://src.fedoraproject.org/rpms/libsepol/blob/master/f/tests/tests.yml libsepol]
| |
| * [https://src.fedoraproject.org/rpms/policycoreutils/blob/master/f/tests/tests.yml policycoreutils]
| |
| * [https://src.fedoraproject.org/rpms/selinux-policy/blob/master/f/tests/tests.yml selinux-policy]
| |
| | |
| Use the `fmf` command line tool to quickly check which tests will be scheduled:
| |
| | |
| > dnf install -y fmf
| |
| > fedpkg clone -a tests/selinux
| |
| > cd selinux
| |
| > fmf ls --filter "tier: 1 | component: checkpolicy"
| |
| /selinux-policy/policy-rpm-macros
| |
| /checkpolicy/sedispol
| |
| /checkpolicy/checkmodule
| |
| /checkpolicy/sedismod
| |
| /checkpolicy/checkpolicy
| |
| /checkpolicy/checkpolicy-docs
| |
| /libsepol/sepol_check_context
| |
| /libsemanage/verify-options-in-semanage-conf
| |
| /libselinux/getsebool
| |
| /policycoreutils/booleans
| |
| | |
| See the Flexible Metadata Format documentation for other options how to [https://fmf.readthedocs.io/en/latest/overview.html#install install] fmf.
| |
