|
|
| (8 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| = Motivation =
| | Moved to: https://docs.fedoraproject.org/en-US/ci/share-test-code/ |
| | |
| In order to make the CI workflow reliable and efficient it is
| |
| crucial to keep the test coverage in a good shape at all times.
| |
| Sharing test code between several packages (even within multiple
| |
| branches of the same package) may significantly help to:
| |
| | |
| * Prevent test code duplication
| |
| * Minimize test maintenance
| |
| * Catch incompatibilities early
| |
| | |
| In general, tests define how the software works and the basic
| |
| functionality of many packages doesn’t change that often. We try
| |
| hard to keep the backward compatibility where possible. Thus it
| |
| seems natural that, for such components, tests guarding the spec
| |
| could change at a slower pace than the distribution branches.
| |
| | |
| See the whole [https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/thread/55U6V6UHA54MJLD2F6JF46EOLMVRUAE7/ ci-list discussion] for some more context.
| |
| | |
| = Implementation =
| |
| | |
| Store test code in your preferred repository and reference the
| |
| tests from the dist-git yaml file. There is also a special
| |
| <code>tests</code> namespace dedicated for storing Fedora CI
| |
| integration tests:
| |
| | |
| * https://src.fedoraproject.org/projects/tests/*
| |
| | |
| Some of the [[CI/Standard_Test_Roles|Standard Test Roles]]
| |
| (currently beakerlib and rhts) support fetching test code from
| |
| remote repositories directly in their config in this way:
| |
| | |
| - role: standard-test-beakerlib
| |
| repositories:
| |
| - repo: "https://src.fedoraproject.org/tests/shell.git"
| |
| dest: "shell"
| |
| | |
| = Examples =
| |
| | |
| Here are some real-life examples where sharing test code can
| |
| increase long-term efficiency.
| |
| | |
| == Shell ==
| |
| | |
| There are several shells which implement the POSIX specification:
| |
| bash, ksh, mksh, zsh, dash. All of them share a significant amount
| |
| of test coverage and it does not make sense to commit & maintain
| |
| identical tests in five different repositories (+ possible
| |
| branches).
| |
| | |
| Shell tests repository:
| |
| | |
| * https://src.fedoraproject.org/tests/shell
| |
| | |
| Bash [https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml tests.yml]:
| |
| | |
| - hosts: localhost
| |
| roles:
| |
| - role: standard-test-beakerlib
| |
| tags:
| |
| - classic
| |
| repositories:
| |
| - repo: "https://src.fedoraproject.org/tests/shell.git"
| |
| dest: "shell"
| |
| tests:
| |
| - shell/func
| |
| - shell/login
| |
| - shell/smoke
| |
| required_packages:
| |
| - expect # login requires expect
| |
| - which # smoke requires which
| |
| | |
| Ksh [https://src.fedoraproject.org/rpms/ksh/blob/master/f/tests/tests.yml tests.yml]:
| |
| | |
| - hosts: localhost
| |
| roles:
| |
| - role: standard-test-beakerlib
| |
| tags:
| |
| - classic
| |
| repositories:
| |
| - repo: "https://src.fedoraproject.org/tests/shell.git"
| |
| dest: "shell"
| |
| tests:
| |
| - shell/func
| |
| - shell/login
| |
| - shell/smoke
| |
| environment:
| |
| PACKAGES: ksh
| |
| SH_BIN: ksh
| |
| required_packages:
| |
| - ksh
| |
| - expect # login requires expect
| |
| - which # smoke requires which
| |
| | |
| == Ruby ==
| |
| | |
| Another example is Ruby: With about 80 packages related to Ruby on
| |
| Rails it would be useful and efficient to have a single place for
| |
| integration tests which verify that the framework is correctly
| |
| working after updating any of these packages. Conversely,
| |
| maintaining those tests in 80 repos would be a tedious task.
| |
| | |
| Currently the shared [[https://src.fedoraproject.org/tests/ruby|tests/ruby]] repository hosts these three
| |
| ruby integration tests:
| |
| | |
| Available integration tests:
| |
| * systemtap-static-probes-in-ruby - exercising ruby's systemtap api
| |
| * bundler-unit-test - run bundler's unit tests
| |
| * run-basic-rails-application - run a simple rails application
| |
| | |
| == SELinux ==
| |
| | |
| Several SELinux user space components are sharing test coverage in
| |
| a single [https://src.fedoraproject.org/tests/selinux.git selinux] test
| |
| repository:
| |
| | |
| * [https://src.fedoraproject.org/rpms/libsepol/blob/master/f/tests/tests.yml libsepol]
| |
| * [https://src.fedoraproject.org/rpms/libselinux/blob/master/f/tests/tests.yml libselinux]
| |
| * [https://src.fedoraproject.org/rpms/libsemanage/blob/master/f/tests/tests.yml libsemanage]
| |
| * [https://src.fedoraproject.org/rpms/libsepol/blob/master/f/tests/tests.yml policycoreutils]
| |
| | |
| = Start =
| |
| | |
| In order to create a new repository in the tests namespace file a
| |
| [https://pagure.io/fedora-infrastructure/new_issue new issue] in
| |
| the fedora infrastructure queue. Specify desired repository name,
| |
| main admin and a short description/justification. For example:
| |
| | |
| * https://pagure.io/fedora-infrastructure/issue/6699
| |
| | |
| Creating new test repositories will be soon supported directly in
| |
| the <code>fedpkg</code> tool as well:
| |
| | |
| * https://pagure.io/fedpkg/issue/176
| |
