|
|
| (7 intermediate revisions by one other user not shown) |
| Line 1: |
Line 1: |
| = Why =
| | #REDIRECT [[CI]] |
| | |
| == Vision ==
| |
| | |
| There are hundreds of packages which make up the Operating System.
| |
| Making sure that they all work together as a whole is not an easy
| |
| task. This becomes even harder as the number of packages and their
| |
| inter-dependencies grows. An extensive testing is required before
| |
| a new version of the operating system is released to ensure it is
| |
| stable enough. That is the past.
| |
| | |
| Imagine an '''Always Ready Operating System''' which consists of
| |
| packages which are constantly kept in a good shape. Integrated
| |
| and stable thanks to an extensive test coverage which is
| |
| continuously executed upon changes in individual packages, in this
| |
| way allowing to prepare a new release in much shorter time, or
| |
| even in no time.
| |
| | |
| Imagine an operating system distribution which you could release
| |
| at any moment. This is where we are heading. Here comes the CI,
| |
| Continuous Integration, as an invaluable tool to ensure everything
| |
| is working together as expected in every point of time.
| |
| | |
| == Manifesto ==
| |
| | |
| Continuous integration aims to ensure broken changes are revealed
| |
| as soon as possible and do not affect other developers, packagers,
| |
| maintainers or users. The feedback that continuous integration
| |
| provides is vital for fast paced agile delivery of software. Late
| |
| testing, long after a change occurs, does not scale to the pace of
| |
| Fedora. Learn about the steps that are crucial for a working
| |
| Continuous Integration in the [[CI/Manifesto|CI Manifesto]].
| |
| | |
| = How =
| |
| | |
| There are three main pieces of the puzzle to get this nicely
| |
| working: A process which clearly defines how to discover and
| |
| execute tests, a set of tools which help to efficiently implement
| |
| the process and the tests themselves.
| |
| | |
| == Process ==
| |
| | |
| In order to clearly distinguish test from the CI system running it
| |
| the [[CI/Standard_Test_Interface|Standard Test Interface]] was
| |
| introduced. It clearly defines essential terms such as test, test
| |
| subject, test suite, test framework, test result, test artifact,
| |
| test system and describes what are their responsibilities and
| |
| requirements.
| |
| | |
| This general approach gives a nice flexibility as it does not
| |
| enforce any specific tools or frameworks to be used. Basically it
| |
| only describes how tests are discovered and where the testing
| |
| results should be stored to be processed by the automation.
| |
| | |
| == Tools ==
| |
| | |
| [[CI/Standard_Test_Roles|Standard Test Roles]] were implemented
| |
| to enable both automation tools and developers in their local
| |
| environments to easily execute tests. This set of ansible roles
| |
| supports various frameworks and allows to execute tests against
| |
| different test subjects (such as classic rpm package, docker
| |
| container or Atomic Host).
| |
| | |
| The testing [[CI/Pipeline|Pipeline]] detects tests for enabled
| |
| packages, executes the test coverage and gathers the results.
| |
| Currently tests are scheduled for new commits, support for pull
| |
| request testing is on the way. Results are displayed in
| |
| [https://src.fedoraproject.org/ Pagure] web interface.
| |
| | |
| == Tests ==
| |
| | |
| The core of the CI success are reliable tests of a good quality,
| |
| well selected, stable, organized and continuously maintained.
| |
| | |
| === Test Types ===
| |
| | |
| In general it makes sense to store tests as close to the upstream
| |
| as possible. So what are the appropriate test types recommended
| |
| for testing the Always Ready Operating System?
| |
| | |
| * Basic functionality tests
| |
| * Integration tests
| |
| | |
| For unit tests it usually makes more sense to store them directly
| |
| within the upstream project repository. However, in some cases it
| |
| might be worth to fetch tests for Fedora CI from the upstream
| |
| repository as well.
| |
| | |
| === Test Code ===
| |
| | |
| Tests are enabled by including the <code>tests.yml</code> file in
| |
| the package dist-git repository as defined by the Standard Test
| |
| Interface. Test code itself can be stored directly in the dist-git
| |
| or fetched from another repository. Shared tests namespace can be
| |
| used for storing test code relevant for multiple packages.
| |
| | |
| === Test Execution ===
| |
| | |
| Executing a test written with the use of Standard Test Roles is as
| |
| simple as running an ansible playbook:
| |
| | |
| ansible-playbook tests.yml
| |
| | |
| However, a set of environment variables needs to be set properly
| |
| in order to execute the test against the desired test subject. See
| |
| [[CI/Tests|Tests]] for more detailed instructions about running
| |
| tests and adding new test coverage.
| |
| | |
| = More =
| |
| | |
| == Contact ==
| |
| | |
| If you have questions or would like to get involved:
| |
| | |
| * IRC channel: <code>#fedora-ci</code> on FreeNode
| |
| * Mailing list: <code>ci@lists.fedoraproject.org</code>
| |
| * Issues: [https://pagure.io/fedora-ci/AtomicCi pagure.io/fedora-ci/AtomicCi]
| |
| | |
| == Links ==
| |
| | |
| Here's a summary of useful links:
| |
| | |
| * [[CI/Standard_Test_Interface|Standard Test Interface]] ... definition of the process
| |
| * [[CI/Standard_Test_Roles|Standard Test Roles]] ... set of ansible roles
| |
| * [[CI/Pipeline|Pipeline]] ... CI pipeline executing the tests
| |
| * [[CI/Tests|Tests]] ... executing and adding tests
| |
| * [https://upstreamfirst.fedorainfracloud.org/ Upstream First] ... tests to be ported to dist-git
| |
| | |
| == FAQ ==
| |
| | |
| * Where do I get the latest Atomic Host images for testing?
| |
