Changes/FedoraDockerRegistry

Fedora Scale-Out Docker Registry

Summary

This is a proposal for a change to the Fedora Infrastructure and Fedora Release Engineering tooling to provide a scalable Docker Registry solution for Fedora that is integrated with the Fedora Docker Layered Image Build Service.

Owner

• Name: Adam Miller and Randy Barlow
• Email: maxamillion@fedoraproject.org and bowlofeggs@fedoraproject.org
• Release notes owner:

Current status

• Targeted release: Fedora 26
• Last updated: 2016-09-07
• Tracker bug: #1345756

Detailed Description

   +--+--------+               +------------------------+
   |  koji     +^--------------+ fedpkg container build +
   +--+---+----+               +------------------------+
      |   ^
      ^   |
   +--+---+----+
   |           |          +----------------------+
   |   OSBS    |          | docker/distribution  |
   |           +---------^+ registry             |
   +-----------+          |                      |
                          | (candidate builds)   |
                          +---------------+-------
                                          |
                                          |
                                          |
                                          |
                                          |
                                          |
                                          |
                               +----------v----------+
                               |                     |
                               | Small Python script |
                               |                     |
                               +----------+----------+
                                          |
                                          |
                                          |
                                          |
                                          |
                                          |
                                          |
+----------------+              +---------v----------+
|                |              | Mirror Manager     |
|  Mirror list   |              | master mirror      |
|                |              +-----------------+--+
+----+----+------+                                |
     ^    |                                       |
     |    |                                       |
     |    |                                       |
     |    |                                       ^
     |    |                             +---------+-------------------+
     |    |                             |                             |
     |    |   +------------------------^+ "Mirror Network"            |
     |    |   |                         | (All our ^olunteer mirrors) |
     |    |   |   +---------------------+                             |
     |    |   |   |                     +-----------------------------+
     |    |   |   |
     |    |   |   |
     |    |   |   |
     |    |   |   |
     |    |   |   |
     |    ^   |   ^
  +--+----+---+---+-+
  | Users           |
  | (docker pull)   |
  +-----------------+

Background

registry: a collection of docker image repositories

repository: named after an image and is a collection of multiple tags of an that image

tag: an arbitrary string assigned to a specific docker image (identified by the image's sha256 checksum) NOTE: The "latest" tag is special and is assumed if no tag is provided. This is true also for a 'docker pull' operation and an image tagged "latest" will be the default image pulled by users.

Proposal

New Tool + MirrorManager[2] + Docker Distribution[3]

• We will build a small Python script that will pull Docker Blobs out of the OSBS registry and put them into mirror manager for distribution to our mirror network.

• MirrorManager is what Fedora uses to manage the public mirror network and distribute content.

• Docker Distribution is the defacto standard open source implementation of the Docker Registry V2 API spec[5]. It provides many features but the ability to have it's back-end storage be provided by a "mirror network" much like the one Fedora has at it's disposal is not one of them. The reason we need this in place is because the mechanism by which you could push a docker image directly to Pulp in Docker Registry v1 no longer exists in v2 so we must instead perform a "sync" operation between the two. (This is a common problem for all known "third party" v2 registry implementations).

Workflow

• OSBS will perform Builds, as these builds complete they will be pushed to the docker-distribution (v2) registry, these will be considered "candidate images". These will be stored in candidate repositories on the docker-distribution registry.
• Testing will occur using the "candidate images" (details of how we want to handle that are outside the scope of this proposal).
• A "candidate image" will be marked stable once it's criteria have been satisfied to do so. (This is vague because this is a topic of ongoing discussion and work to decide what criteria an image will need to abide by before being considered "stable" and promoted as such)
• Once stable, the images will be pushed into stable repositories in the docker-distribution registry.
• The new Python tool will split that content and sync the image layers along with their metadata to Mirror Manager master mirror. It will also sync the repo metadata to somewhere Mirror List can pick it up.
• Mirror Manager will distribute to the mirrors the image layers.
• The docker clients will request Manifests from Mirror List. Mirror list will return schema 2 Manifests that contain lists of URLs to the mirrors where the client can retrieve the blobs.

General Notes

A couple of things to note about maintenance and uptime considerations:

The Intermediate docker-distribution registry is needed for builds in koji+OSBS

All new components in this design should be able to be locked down, similar to the "Fedora internal" components like koji (builders, etc) and bodhi (signing, etc).

• [2] https://github.com/fedora-infra/mirrormanager2/
• [3] https://github.com/docker/distribution/
• [4] https://docs.docker.com/registry/spec/api/

Benefit to Fedora

This will allow for Fedora to provide packages, software, and other content in the form of a Docker Image as an officially released artifact from the Fedora Project that is released and hosted much in the same way RPMs are today. These images can then be included in the distribution in various ways. This could potentially be used by the Modularization effort or by any other part of the Fedora.next initiative that may arise.

Scope

Proposal owners

Proposal owners shall have to:

• Implement the proposed Design of a Scaled-Out Docker Registry
  • Deploy Pulp
  • Deploy Crane
  • Deploy Docker-Distribution Registry
  • Integrate with MirrorManager for content distribution
• Document the system

Task matrix

This is a RACI matrix for tasks required to implement the RelEng Automation Workflow Engine. Work is tracked in Taiga: http://taiga.cloud.fedoraproject.org/project/acarter-fedora-docker-atomic-tooling/wiki/home

Is this current?

It is, as of 2016-09-07

Definitions

Here, we're using what Wikipedia calls "RACI (alternative scheme)":

Responsible

The person responsible for the performance of the task. There should be exactly one person with this assignment for each task.

Assists

Those who assist completion of the task.

Consulted

Those whose opinions are sought; and with whom there is two-way communication.

Informed

Those who are kept up-to-date on progress; and with whom there is one-way communication.

Task Table

This an early cut. Please feel free to add new tasks as appropriate — just let one of the coordinators know.

Task	Subtask	Responsible	Assists	Consulted	Informed	Current Status
Implement the proposed design of a Scaled-Out Docker Registry		Adam Miller				0%
	Deploy solution, including ansible playbooks added for Fedora Infrastructure Ansible repo	Adam Miller				0%
	Deploy Pulp	Adam Miller				0%
	Deploy Crane	Adam Miller				0%
	Deploy docker-distribution registry	Adam Miller				0%
	Integrate with MirrorManager for content distribution	Adam Miller				0%
Document the system		Adam Miller				0%

Glossary of Nicknames

• maxamillion Adam Miller
• bowlofeggs (rbarlow) Randy Barlow

Functional Requirements

The following features are functional requirements

• Users must be able to perform a

  docker pull registry.fedoraproject.org/fedora

  and have the actual image layer data come from a local mirror via mirrormanager.

Other developers

• (anything here)?

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

Once the service is deployed, users can perform the following on their systems to test.

$ dnf -y install docker
$ systemctl start docker
$ docker pull registry.fedoraproject.org/fedora

N/A (not a System Wide Change)

User Experience

N/A (not a System Wide Change)

Dependencies

N/A (not a System Wide Change)

Contingency Plan

• Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
• Contingency deadline: N/A (not a System Wide Change)
• Blocks release? No (not a System Wide Change)
• Blocks product? N/A

Documentation

FIXME