CI/Share Test Code

Share Test Code

Motivation

In general, tests define how the software works and the basic functionality of many packages doesn’t change that often. We try hard to keep the backward compatibility where possible. Thus it seems natural that, for such components, tests guarding the spec could change at a slower pace than the distribution branches.

There are several shells which implement the POSIX specification: bash, ksh, mksh, zsh, dash. All of them share a significant amount of test coverage and it does not make sense to commit & maintain identical tests in five different repositories (+ possible branches).

Another example is Ruby: With about 80 packages related to Ruby on Rails it would be useful and efficient to have a single place for integration tests which verify that the framework is correctly working after updating any of these packages. Conversely, maintaining those tests in 80 repos would be a tedious task.

See the whole ci-list discussion for some more context.

Examples

Shell tests repository:

• https://src.fedoraproject.org/tests/shell

Bash tests.yml:

- hosts: localhost
  roles:
  - role: standard-test-beakerlib
    tags:
    - classic
    repositories:
    - repo: "https://src.fedoraproject.org/tests/shell.git"
      dest: "shell"
    tests:
    - shell/func
    - shell/login
    - shell/smoke
    required_packages:
    - expect            # login requires expect
    - which             # smoke requires which

Ksh tests.yml:

- hosts: localhost
  roles:
  - role: standard-test-beakerlib
    tags:
    - classic
    repositories:
    - repo: "https://src.fedoraproject.org/tests/shell.git"
      dest: "shell"
    tests:
    - shell/func
    - shell/login
    - shell/smoke
    environment:
      PACKAGES: ksh
      SH_BIN: ksh
    required_packages:
    - ksh
    - expect            # login requires expect
    - which             # smoke requires which