|
|
| (6 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| = Introduction =
| | Moved to: https://docs.fedoraproject.org/en-US/ci/quick-start-guide/ |
| | |
| Are you eager to try out how the Fedora CI tests work? Do you want to get a quick hands-on experience without having to read too much documentation? This quick introduction for the impatient will show you a minimal set of steps to execute existing tests as well as provide useful links to resources where you can learn more.
| |
| | |
| = First Steps =
| |
| | |
| Install the following essential packages on your system:
| |
| | |
| sudo dnf install fedpkg standard-test-roles
| |
| | |
| Use <code>fedpkg</code> to clone the package git repository. See the [[Package_maintenance_guide|Package Maintenance Guide]] for more info about the tool.
| |
| | |
| fedpkg clone -a bash
| |
| | |
| Tests are defined according to the [[CI/Standard_Test_Interface|Standard Test Interface]] in the <code>tests</code> directory:
| |
| | |
| cd bash/tests/
| |
| | |
| Test coverage to be executed together with the basic set of metadata is described in the [https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml tests.yml] playbook. Use <code>ansible-playbook</code> to run all available tests for the classic environment on the local host:
| |
| | |
| ansible-playbook --tags=classic tests.yml
| |
| | |
| From the ansible output you can directly see an overall summary of the testing. If you see <code>failed=0</code> at the end of the log then all tests passed:
| |
| | |
| localhost: ok=29 changed=11 unreachable=0 failed=0
| |
| | |
| For more detailed test results check the <code>test.log</code> and other files in the <code>artifacts</code> directory:
| |
| | |
| vim artifacts/test.log
| |
| | |
| That's it! You just executed test coverage for the Bash package :)
| |
| | |
| = Test Subjects =
| |
| | |
| To execute tests against different test subjects we need to prepare the environment. Let's store the detailed test results in <code>/tmp/artifacts</code>, use dynamic inventory as defined by the [[CI/Standard_Test_Roles|Standard Test Roles]] and download the latest Atomic Host image.
| |
| | |
| export TEST_ARTIFACTS=/tmp/artifacts
| |
| export ANSIBLE_INVENTORY=/usr/share/ansible/inventory
| |
| curl -Lo /tmp/atomic.qcow2 https://getfedora.org/atomic_qcow2_latest
| |
| | |
| Now let's try to run tests against all supported test subjects.
| |
| | |
| == Classic ==
| |
| | |
| Run tests against classic rpms installed on the system:
| |
| | |
| export TEST_SUBJECTS=''
| |
| ansible-playbook --tags=classic tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Classic|Classic]] for detailed docs.
| |
| | |
| == Container ==
| |
| | |
| Run tests in a docker container:
| |
| | |
| export TEST_SUBJECTS=docker:docker.io/library/fedora:latest
| |
| ansible-playbook --tags=container tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Container|Container]] for detailed docs.
| |
| | |
| == Atomic ==
| |
| | |
| Run tests against the Atomic Host:
| |
| | |
| export TEST_SUBJECTS=/tmp/atomic.qcow2
| |
| ansible-playbook --tags=atomic tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Atomic|Atomic]] for detailed docs.
| |
| | |
| = Hints =
| |
| | |
| == Debug ==
| |
| | |
| Would you like to investigate why a test failed? Enable debugging to easilly connect to running Atomic or Container to investigate:
| |
| | |
| export TEST_DEBUG=1
| |
| ansible-playbook --tags=atomic tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Debug|Debug]] for details about debugging.
| |
| | |
| == Ignore ==
| |
| | |
| Use <code>.gitignore</code> to specify files that Git should ignore. Such files are created during tests run. Create a <code>tests/.gitignore</code> file with the following contents:
| |
| | |
| # Ignore tests runs/artefacts.
| |
| artifacts/**
| |
| **/*.retry
| |
| | |
| = Contribute =
| |
| | |
| Are you interested in contributing a new test coverage? You are most welcome! As you have seen [[CI/Tests#Executing|Executing]] a test is quite easy. [[CI/Tests#Writing|Writing]] a new test or [[CI/Tests#Wrapping|Wrapping]] an existing one is quite simple as well. Here's a few recommendations for creating a new pull request.
| |
| | |
| == Fork ==
| |
| | |
| Unless you are maintainer of the package, who has direct commit access, create a fork of the package git repository using the Fork button in [https://src.fedoraproject.org/rpms/bash Pagure] web interface and add your private fork as a new remote. Create a branch for your new tests. For example:
| |
| | |
| git remote add fork ssh://psss@pkgs.fedoraproject.org/forks/psss/rpms/bash.git
| |
| git checkout -b tests
| |
| | |
| If you are not a Fedora packager, use <code>fedpkg</code> command to clone you fork and set up the git repo config so that you are able to push to it. See [[CI/Pull_Requests|Pull Requests]] for more detailed info.
| |
| | |
| fedpkg clone -a forks/psss/rpms/bash
| |
| git checkout -b tests
| |
| | |
| == Add ==
| |
| | |
| Create new test coverage under the <code>tests</code> directory, update the <code>tests.yml</code> file accorgingly or create a new one. Run tests and verify they are stable and working fine in all supported environments. Add files to git, commit and push:
| |
| | |
| git add tests.yml test1 test2 test3
| |
| git commit -m "Add CI tests using the Standard Test Interface"
| |
| git push fork tests:tests
| |
| | |
| It is a good idea to include more details and links in the commit message to make the pull request easier for review:
| |
| | |
| Add CI tests using the Standard Test Interface
| |
|
| |
| Adding initial set of basic functionality tests for bash
| |
| according to the Standard Test Interface [1]. See Quick Start
| |
| Guide for brief introduction about how to run these tests [2].
| |
|
| |
| [1] https://fedoraproject.org/wiki/CI/Standard_Test_Interface
| |
| [2] https://fedoraproject.org/wiki/CI/Quick_Start_Guide
| |
| | |
| Create a new pull request from your <code>tests</code> branch against the master branch in the [https://src.fedoraproject.org/fork/psss/rpms/bash Pagure] web interface. You might want to include an additional info about the tests such as:
| |
| | |
| There are three tests available: smoke and func have been tested
| |
| across all environments (classic, container, atomic), login is
| |
| relevant for classic only (because of a missing dependency).
| |
| Please, merge the tests into all currently supported branches.
| |
| | |
| == Results ==
| |
| | |
| Once the pull request is created CI Pipeline will detect it and execute tests. Once the test execution is finished you will see results of the testing on the pull request page. See the [[CI/Pipeline|Pipeline]] page for the list of active pipelines and result examples.
| |
