From Fedora Project Wiki

Objective: Deliver Fedora Atomic Host via CI/CD

Goal

We aim to deliver Atomic Host in Fedora via Continuous Integration and Continuous Delivery.

A continuous integration pipeline builds and composes Fedora Atomic Host artifacts exactly as they will be delivered and used (eg: in QCow2 and OSTree formats), and runs integration tests on it in that form.

Continuous integration pipeline will gate changes to packages. It will prevent a broken package change, as defined by integration tests, from affecting other developer's and/or testers of Fedora. The package maintainer will be able to change the tests on which their package is gated. The pipeline also gates broken combinations of package changes.

Continuous delivery will deliver the artifacts composed during integration step above to users of Fedora Atomic Host. It allows, but does not mandate constant or frequent delivery of changes to the Fedora mirrors. During delivery we can prevent unacceptable changes, due to policy or schedule, from reaching users.

More Details

Deliverables and Scope

During this objective, the focus and scope of CI will be on packages included in Atomic Host. These represent many of the core packages in Fedora. Testing and gating of build requirements is out of initial scope, and should be covered by future objectives.

In addition the delivery will be limited to composed artifacts of Atomic Host:

  • QCow2 images
  • Vagrant images
  • OSTree trees
  • Amazon Public Cloud images

The composes delivered by the CI/CD pipeline will not include full RPM based composes of Fedora at this point. Although the integration tests will help improve the quality of Fedora overall.

Key Results

Look for the following key results during progress on this objective:

  • 100 packages in Fedora Atomic host have at least one test stored in dist-git according to the spec.
  • Work with package maintainers to start to add additional tests or update tests when needed.
  • 100% of tests in Fedora Atomic Host packages pass on OSTree composes before a composed Fedora Atomic Host image is delivered.
  • Fedora Atomic Host artifact composes have no human interaction after the gating points.
  • Tests ported upstream from downstream sources have been reviewed by QE.
  • Tests check that Fedora Atomic Host works with key containers produced by Modularity effort.

Plans and Work Areas

Changes to how Atomic Host is built are needed. We have identifed several requirements that are needed for the CI/CD pipeline:

Requirements:

  • The definition of what is composed into Atomic Host artifacts should include specific versions of packages, and all dependencies included.
  • The definition of what is composed in an Atomic Host should be stored in a git repository so that changes can be detected easily. The CI/CD pipeline can be triggered off of changes to this repository.
  • A mechanism to make a future composed Atomic Host artifact, contain an earlier (in RPM NVR parlance) version of a package.
  • The high level functional Atomic Host tests should live in the same git repository with the rigorous definition of what goes into an Atomic Host.
  • A mechanism to tell the CI Pipeline that multiple dist-git repository changes (i.e. multiple changing RPMs) should be built and tested together.

It is possible the build and MBS parts of Modularity could be used to accomplish these requirements. Experimentation is needed in order to determine if there are any blockers for Atomic Host being built from a module.

CI Working Group

Much of the pipeline and testing effort on this Objective will be undertaken by a new "CI Special Interest Group". This group will invite people from the Fedora Engineering team, Fedora Release-Engineering team, Fedora QA, Security Team and more. Work will be coordinated when possible with the Factory 2.0 effort. Contact info:

Atomic Host Working Group

This effort is co-owned by the Fedora Atomic Working Group. The content and deliverables of Atomic Host affect the manner in which it is tested and delivered.

Objective Lead

  • Dominik Perpeet - Facilitating test generation, pipeline creation and wiring into pipeline

Dominik will work closely with Dusty Mabe on Atomic Host and Atomic Working Group communications, build tooling and wiring

Timeframe

This Objective aims to deliver Fedora Atomic Host in parallel with the current delivery methods. At a point at or around the Fedora release, we would aim for a cut over to using the CI/CD pipeline for delivery.

By Flock in September 2017, we will discuss the concrete results of this objective so far, and together imagine next steps for CI/CD objectives in Fedora.

Retrieved from "https://fedoraproject.org/w/index.php?title=Initiatives/Continuous_Integration_and_Delivery_of_Fedora_Atomic_Host&oldid=700472"