From Fedora Project Wiki

< User:Lmacken
Revision as of 17:55, 13 November 2015 by Lmacken (talk | contribs)

Blockers / dependencies for redoing the Atomic workflow (try to prioritize these projects):

Things that are not blocked (add to the short term epic):

  • better test case coverage
  • better ostree management
  • Single Atomic Host OSTree repository. https://fedorahosted.org/rel-eng/ticket/6125
  • creating new ostrees starts with an empty tree and merges this into master if there are changes
  • tagging, garbage collection, etc
  • automated build, compose, test, release process for all artifacts: pxe2live, AMIs, Installer ISO, GCE (?)

Functional Requirements

  • Automated build/compose pipeline
  • Atomic Host build artifacts should only be refreshed/rebuilt when something in their package manifest changed/updated in a new compose
  • Automated test pipeline
  • All Atomic Host build artifacts should be automatically tested with an acceptable level of test coverage that we trust the resulting "successful" marked artifacts are ready for release
  • Automated Release pipeline
  • Once all of the artifacts pass the automated tests, they are then automatically signed and released.
  • Websites
  • The Cloud download website should be automatically updated with the new content as it's released. https://stg.getfedora.org/en/cloud/download/atomic.html
  • Updates
  • Updates for the components of Atomic Host's OSTrees should have automated tests in taskotron
  • Updates for the OSTree of Atomic Host should be refreshed as soon as one of the components of it's manifest hit's the Fedora or Fedora Updates stable repository.

"Releng-o-tron" (for lack of a better name)

A fedmsg consumer that listens for changes, queries PDC for products that are affected, and triggers composes & automatic tests accordingly.

Updates/updates-testing Workflow

  • Package maintainer commits package changes to git, builds it in koji, and submits an update to bodhi
  • Taskotron automatically kicks off the RPM-based tests
  • Updates are signed (currently by hand, which should eventually be automated)
  • Bodhi "pushes" the update in a batch with others, which currently involves mashing a repository and composing the Atomic OSTrees.
  • Ideally, we want to create koji tasks for both mash and ostree composes, and have bodhi simply send a single fedmsg with the batch of updates listed, and wait for releng-o-tron to do the coordination.
  • Releng-o-tron picks up the fedmsg with all of the updates for the push
  • Triggers mashing updates/updates-testing tasks in koji
  • For each update in the batch, determine which artifacts are affected by querying PDC
  • Upon completion of the repositories, trigger artifact composes against them
  • Send a fedmsg upon completion. Really, it will store the results in a releng-o-tron "resultsdb" and the resultsdb is already instrumented to send a fedmsg.
  • Task-o-tron picks up the fedmsg and tests the artifacts, and kicks off tests accordingly
  • Tests OSTree, ISO, AMI, etc
  • After Taskotron gives everything the "thumbs-up" (via fedmsg), releng-o-tron symlinks/rsyncs everything live, and then sends a fedmsg
  • Bodhi picks up the fedmsg and "finishes" the push (sends emails, updates/closes bugs, comments on updates, etc)
    • A question: do we then only "finish" a push after compose artifacts are completed? That's new territory if so. We currently only finish a push when both the mash and rpm-ostree are completed anyway. So it's already an "all or nothing" model.

Rawhide Gating

  • Packagers build packages in a 'rawhide-pending' tag, or something like that. We disallow building in the traditional rawhide tag directly to enforce this.
    • We could potentially streamline the workflow of getting new upstream releases into rawhide and tested with anitya->new-hotness
  • koji publishes a message when a new rawhide-pending build is built.
  • qa taskotron notices this and runs some series of integrity checks on rawhide-pending. It publishes a message about this.
  • If good, releng-o-tron notices qa-taskotron's result and it moves all the rawhide-pending builds into rawhide-proper (or whatever its called).
  • If bad, it moves suspect builds from rawhide-pending into rawhide-badnewsbears.
  • qa-taskotron can be set up to notice this as well, and start the integrity-check again.
  • We can configure FMN to send emails to packagers when their packages go to rawhide-badnewsbears.

Requirements

  • releng-o-tron should know how to compose everything, ideally with a simple plugin framework that can allows us to easily add new artifacts.
  • releng-o-tron should be the final gate-keeper of all of the bits
  • It should perform/trigger the syncing of bits to the master mirror, instead of relying on a cron job to do so.
  • We should ensure no fedmsgs ever get dropped (via gilmsg)
  • Along with knowing how to compose OSTrees, it should also be able to handle merging, tagging, and archival/garbage collection of branches.
  • Streamline the End Of Life process. With a single command/click of a button it should update the pkgdb, update bugs, carve out OSTrees, and archive content.

Questions

  • Do we want to use the Bodhi web interface to list all of the "testing" compose artifacts and facilitate community testing/feedback/karma?

Concerns

  • We may need to rethink how we push bits to the users. With our current mirroring setup, there are some mirrors that only pull twice a day. If we're hoping to asynchronously push bits out all day, we might want to investigate a proper CDN setup.
Retrieved from "https://fedoraproject.org/w/index.php?title=User:Lmacken/AtomicWorkflowDraft&oldid=427964"