Container:Guidelines

Container Maintainer Guidelines

In the Fedora world, the concept of being a Package Maintainer is well known as all the software currently released and published as an official "Build Artifact" of the Fedora Project for inclusion in the Fedora GNU/Linux distribution has always been packaged in RPM Package Manager Format.

However, as technology changes so must the Fedora Project. The concept of "containers" on Linux has become quite prominent and Fedora will be publishing container images as officially released Build Artifact. One thing to note is that container images are not a new software packaging format but more so a delivery mechanism where many different things can be easily shipped as a single unit. An example of this is packages that can be combined to deliver an easily ran "software solution".

Below you will find Guidelines similar in nature to that of the Fedora Packaging Guidelines but catered towards the concept of Containers. Fedora is targeting the any OCI Compliant container runtime implementation.

General Fedora Container Information

In this section you will find general information about Fedora Container Images that should be useful for Fedora Container Layered Image Maintainers.

Build System

In order to get a better understanding of the big picture of how all this works, Container Maintainers might find the Layered Image Build Service Architecture Document interesting. However, extensive coverage of the Build System is out of the scope of this Guidelines document.

Vocabulary Terms

• Container Registry - (often refereed to as simply a "registry") a service that stores and distributes Container Images via namespaces/repositories.
• Fedora Generational Core - defined by the Fedora Modularity effort as a part of the Base Runtime and will serve as our registry namespace. For all intents and purposes this will be a synonym of the Fedora Release. (Will often be noted as $FGC)

Fedora Container Naming

A Fedora Container Layered Image name should be the same as the the name of main service that it intends to provide end users. Therefore, naming must follow the Fedora Naming Guidelines.

Container naming as it will exist in Fedora Package DB (and [Distgit http://pkgs.fedoraproject.org/cgit/container]) should be relatively standard. None of the Fedora releases/DistGit-branch naming should be taken into consideration by the main container name, just as it is for RPM Package Naming.

Fedora content is now "namespaced" in both Fedora Package DB and DigtGit, with the default namespace being 'rpms' for backwards compatibility. This allows for Container Layered Images to share the same base component name as their RPM counterpart (where applicable).

Below is a reference of the Registry Layout which will clarify how different Fedora releases are handled.

Fedora Base Image

The Fedora Base Image provides information that can be used by the Layered Images via inherited Environment Variables.

These are outlined below:

• $FGC is defined as Fedora Generational Core by the Fedora Modularity effort as a part of the Base Runtime and will serve as our registry namespace.
• $DISTTAG is defined just as it is for RPMs, but since Dockerfiles lack a mechanism similar to RPM Macros this is being stored in the base image such that it can be inherited by layered images.

Fedora Container Registries and Updates

In Fedora there are two Registries: candidate and stable.

All Layered Image Builds end up in the candidate registry as soon as they are successful in the Fedora Layered Image Build System. These images can immediately be pulled (example using docker: docker pull candidate-registry.fedoraproject.org/$FGC/$NAME:latest).

Gated releases will happen on a Two Week Cadence, alternating with the Fedora Two Week Atomic Host.

Registry Layout

Fedora Base Images will be available at the "root" namespace of the registry, an example is below:

https://candidate-registry.fedoraproject.org/fedora:24
https://candidate-registry.fedoraproject.org/fedora:25
https://candidate-registry.fedoraproject.org/fedora:latest

https://registry.fedoraproject.org/fedora:24
https://registry.fedoraproject.org/fedora:25
https://registry.fedoraproject.org/fedora:latest

Fedora Layered Images will be available in their respective $FGC namespace which coorelates to their DistGit branch and Koji tag. An example is as follows for the f25 Fedora Generational Core and the cockpit container image.

There are multiple tags applied to each image:

• $FGC/Name:Version-Release (including $DISTTAG)
• $FGC/Name:Version
• $FGC/Name:latest
  • The :latest tag can be omitted when issuing a docker pull command.

The latter two tags are updated in-place and a new execution of docker pull will get the latest image.

https://candidate-registry.fedoraproject.org/f25/cockpit:0-1.f25docker
https://candidate-registry.fedoraproject.org/f25/cockpit:0
https://candidate-registry.fedoraproject.org/f25/cockpit:latest

https://registry.fedoraproject.org/f25/cockpit:0-1.f25docker
https://registry.fedoraproject.org/f25/cockpit:0
https://registry.fedoraproject.org/f25/cockpit:latest

Searching the Registry

At the time of this writing there is no search functionality for the Container Registry but we will release a manifest of all available stable Fedora Layered Images in the Fedora Container Registry as Layered Images become available for release. We aim to change this in the future and provide search functionality to users.

Container Guidelines

The Container Guidelines are a collection of common issues and the severity that should be placed on them. While these guidelines should not be ignored, they should also not be blindly followed. If you think that your container should be exempt from part of the Guidelines, please bring the issue to the Fedora Container Committee (Pending Existence). In the absence of a Fedora Container Committee, please seek guidance from the Fedora Cloud SIG.

Containers (Dockerfile)

Container images in Fedora are built using a Dockerfile much in the same way an RPM is built using a spec file. In this section are Fedora Guidelines for creating Container images using a Dockerfile.

Dockerfile Guidelines Upstream
These guidelines are a Fedora adaptation of the Upstream Project Atomic effort to define Container Best Practices.

FROM

As defined by the Dockerfile reference, the FROM instruction must be the first line of a Dockerfile. The FROM instruction must be fully-qualified with a registry name, image name, and tag as shown in this example:

FROM registry.example.com/imagename:tag

This provides a guarantee of where the base image is coming from when being built by the build service or when rebuilt by a user.

For most layered images built by the Fedora Layered Docker Image Build Service, the FROM line will use one of the Fedora base images that exist on the Fedora Registry:

FROM registry.fedoraproject.org/fedora:25

It is also possible to use another layered image as the base layer, as in this example:

FROM registry.fedoraproject.org/f25/kubernetes-master:latest

LABELS

Dockerfiles have a concept of a LABEL which can add arbitrary metadata to an image as a key-value pair. Fedora Guidelines on the topic of LABELs follows the Project Atomic Container Application Generic Labels standards for LABEL definition.

Required LABELs for a Fedora Layered Image are as follows:

Name	Description
com.redhat.component	The Bugzilla component name where bugs against this container should be reported by users.
name	Name of the Image
version	Version of the image
release	Release Number for this version
architecture	Architecture the software in the image should target (Optional: if omitted, it will be built for all supported Fedora Architectures)
run or usage	Either provides an Atomic run line, or a human readable example of container execution
help	Provides a detailed narrative of how the container is meant to be used. In many cases will be a link.

Optional labels for Fedora Layered Images

Name	Description
install	Powers "atomic install" command. Not used for system containers.
uninstall	Powers "atomic uninstall" command. Required if Install is present.
Generics	Any of the Container Application Generic Labels which are appropriate to the container, such as "stop", "debug", "url", or "changelog-url"
atomic.type	Used for system containers, see below.

See LABEL SPECIFICATION below for more details on what's required for each of these labels.

Dockerfile Label Guidelines Upstream
The LABELs used here are meant to be a Fedora adaptation of the upstream Project Atomic effort to define Container Application Generic Labels as well as Container Best Practices.

These LABELs should be defined in a single line of the Dockerfile such that they don't each lead to another layer in the build. The following is a very simple Dockerfile example containing the required LABELs:

A recommended pattern is to define these items as ENV variables such that they can be used elsewhere, also note the $FGC and $DISTTAG.

$FGC is defined as Fedora Generational Core by the Fedora Modularity effort as a part of the Base Runtime and will serve as our registry namespace.

$DISTTAG is defined just as it is for RPMs, but since Dockerfiles lack a mechanism similar to RPM Macros this is being stored in the base image such that it can be inherited by layered images.

By following the pattern below, we can define the container specific information in one place on the ENV line and have it be set properly in the LABEL line (again, noting the $FGC and $DISTTAG being used but never defined as these are inherited).

FROM registry.fedoraproject.org/fedora:25
LABEL MAINTAINER "Adam Miller" <maxamillion@fedoraproject.org>

ENV NAME=myawesomecontainer VERSION=0 RELEASE=1 ARCH=x86_64
LABEL   com.redhat.component="$NAME" \
        name="$FGC/$NAME" \
        version="$VERSION" \
        release="$RELEASE.$DISTTAG" \
        architecture="$ARCH" \
        usage="docker run -p 9000:9000 f25/myawesomecontainer" \
        help="Runs myawesomeapp, which listens on port 9000 and tells you how awesome it is.  No depedancies." 

LABEL SPECIFICATION

Some additional details about how each label is to be populated.

com.redhat.component: Existing Bugzilla component against which bugs in this image should be reported.

name: Name of the image. If the image replaces a standard RPM, it should have the exact same name of that RPM. Otherwise, please see naming guidelines above.

version: Usually 0. Populated from the ENV variable. See "VERSIONING" below for explanation.

release: Populated from the ENV variable. Should start at 1, and increment every time the image changes. Generally carries a suffix of "f##container", depending (i.e. "f26container").

architecture: usually "x86_64", unless the container image supports other/all architectures.

usage: a human-readable example command line for invoking the container. Required if run is not present. Should include all likely options, such as ports, volumes, and any required command-line parameters. You may use any container runtime as your example. Example from the OwnCloud container:

   usage="docker run -d -P -v owncloud-data:/var/lib/owncloud -v owncloud-config:/etc/owncloud owncloud"

run: a command line to invoke the container, suitable for use by the Atomic CLI, including placeholders and the embedded atomic-run code. Must successfully execute on a suitable Fedora Atomic system. Required if "usage" is not present. Example for the Cockpit container:

   run="/usr/bin/docker run -d --privileged --pid=host -v /:/host IMAGE /container/atomic-run --local-ssh"

install: If your container requires preparation of the host system before the container can be run then using the install label that is identified and used by the Atomic CLI can be useful. Please refer to the upstream documention for more information.

uninstall: If you have an install label please provide an uninstall label as well in order to delete any files and/or to clean up any configuration that was done or to the host system. It is not required to delete files that may contain user data. In unusual cases there may be no files or configuration to clean up from the install label so the uninstall label might not be needed. Please refer to the upstream documentation for more information.

help: Required. Either a short description of how to run and use the container, or a link to a longer help file. See Help File below for more details.

VERSIONING

In the previous section there was coverage of LABELs, one of those is the Version that is set in the example using the ENV variable VERSION which at this time needs to be 0. Any changes made to the Dockerfile or the contents of the resulting image that would require a rebuild should increment the Release LABEL that is similarly set using the ENV variable named RELEASE. The reasoning for this is described below.

At this time there is no way to automatically populate the Version/VERSION value with the same value of the latest version of the primary RPM belonging to the container image. This is something that is currently on the roadmap.

Why is this needed?

If we set the Version LABEL to the version of it's respective RPM at the time of the Container Image Review, then the maintainer will constantly have to update it by hand every time there is a RPM update which is inconvenient and error prone. Beyond that, there's a possibility that the version of the RPM could be updated by the layered image automatic rebuilds and the maintainer isn't able to update the Dockerfile in a timely manner (Automatic Rebuilds are done by Release Engineering in order to pull in security updates for all layered images). If this were to happen, then the version of the container image will not match the version of the software it's meant to deliver which would lead to confusion and potentially unexpected negative side effects for users. Therefore, for the time being we're saying that the version number of the container is not meaningful but it will be as soon as possible.

Help File

Just like traditional packages, containers need some "man page" information about how they are to be used, configured, and integrated into a larger stack. For simpler containers, like utility libraries, this can probably be only a couple of lines. But for sophisticated applications, a larger Help File will be needed. This Help File must be supplied as part of the Container Review, and must have one of the three following names:

• help1.txt
• README
• README.md

Maintainers should include a copy of this help file in the base directory of their images.

The help file should contain all of the following, depending on the requirements of the image:

• A brief description of what service/software the image contains.
• What purpose it fulfills in a larger infrastructure, if any.
• If it is possible to configure the contained service the file must contain directions on how to do so.
• If the container has any dependencies on other services (for example a database) the file must detail these.
• If the container uses any volumes the file must detail what each one is for, see VOLUMES guidelines for more detail.
• An explanation of each PORT the image listens on, including its protocol and purpose.
• Links to any external documentation or software project pages, if such pages exist.
• If the container has any special requirements (like lots of RAM, or sound server access), these must be listed.
• If the application has major variants on how it can be built, information about these (e.g. mod_php vs. fastcgi) is required.

Example Help File:

 PostgreSQL 9.6 image, for supplying PostgreSQL within a Kubernetes stack.
 Includes all of "contrib", but does not include PostGIS or any automated replication management.
 
 Volumes:
 pg-config: usually mapped to /etc/postgresql/, contains the configuration files for PostgreSQL, 
   including postgresql.conf and pg_hba.conf. Read-only within the container.  
   Should map to local storage.
 pg-data: usually mapped to /var/lib/pgsql/data, contains database data.  Should be stored in 
   a persistent volume or network storage of some kind.  Requires high-throughput, read-write storage.
 
 Ports:
 5432: the PostgreSQL data port using the pgsql protocol.
 
 Documentation:
 See https://www.postgresql.org/docs/9.6/static/index.html
 
 Requirements:
 This is a transactional database, and requires 256MB of RAM and at 
 least one dedicated core.  If heavily used, can require much greater system resources.
 
 Configuration:
 Sample postgresql.conf and pg_hba.conf files are supplied in the image.  For further documentation, 
   please see the PostgreSQL docs.  Note that these files must be readable by the "postgres" user 
   inside the container.
 The database system is initialized with the admin user/password of postgres/postgres.  If you wish
   to override this, supply the ENV vars ADMIN_USER and ADMIN_PASS inside the container at initialization time.

CMD / ENTRYPOINT

Another item required is a CMD or ENTRYPOINT entry so that when an user were run perform the following command (for example), expected behavior occurs.:

docker run registry.fedoraproject.org/f25/myawesomecontainer

For more information on these entries, please reference the upstream Dockerfile documentation. The following is extending on the above example, showing a CMD directive.

FROM registry.fedoraproject.org/fedora:25
LABEL MAINTAINER "Adam Miller" <maxamillion@fedoraproject.org>

ENV NAME=myawesomecontainer VERSION=0 RELEASE=1 ARCH=x86_64
LABEL   com.redhat.component="$NAME" \
        name="$FGC/$NAME" \
        version="$VERSION" \
        release="$RELEASE.$DISTTAG" \
        architecture="$ARCH"

CMD printf "My Awesome Container!\n"

Volumes

The use of container volumes for persistent data is permitted and encouraged, but the following guidelines need to be followed:

• Any user data that would be at risk of loss on update must be in a volume.
• Any application configuration data that requires persistence must be in a volume. Configuration by environment variables instead is also allowed, either together or instead of configuration volumes.
• All volumes listed in the Dockerfile must be listed in the Help File.
• The example run command should have the volume with a persistent name (e.g. "docker run -d -v owncloud-data:/var/lib/owncloud -v owncloud-config:/etc/owncloud owncloud")
• Volumes must be defined as narrowly as possible. Specifically, unless the image is intended for use as a system container intended for system administration, volumes must be defined so as to mount system directories which are exclusive to the container. For example, the container must mount /etc/application-name/ for config files, not /etc/.

Each volume in the Help File must have the following:

• The full path of the volume
• Why it is marked a volume (such as why this config needs persistence or indicating user data lives there)

Volumes listed in the Help File should also include information about space, permissions, and performance requirements.

The readme may contain suggested additional volumes that aren't made mandatory by the Dockerfile, such as locations for generated, rather than self signed, ssl certificates.

System Containers

System containers are read-only, OS specific runc containers running as systemd services. They use OSTree as storage, skopeo to pull from registries and are managed with the atomic CLI.

System containers' Dockerfiles have the following differences:

LABELS

• install/uninstall: system containers should not have an install/uninstall label. The install is always done through atomic install --system $IMAGE which checks out the image and creates a systemd service.

• run/usage: system containers should not have a run/usage label. All system containers, post installation, should be ran with systemctl start $SERVICE or atomic run $CONTAINER.

• atomic.type: Include this as a label if the container is meant to be ONLY ran as a system container. This is a special label to tell the atomic CLI that the image should be pulled to ostree. Usage: atomic.type='system'.

An example set of labels for a system container:

ENV VERSION=0.1 RELEASE=1 ARCH=x86_64
LABEL com.redhat.component="flannel" \
      name="$FGC/flannel" \
      version="$VERSION" \
      release="$RELEASE.$DISTTAG" \
      architecture="$ARCH" \
      summary="An etcd driven address agent, intended to be run as a system container" \
      maintainer="Giuseppe Scrivano <gscrivan@redhat.com>" \
      atomic.type='system'

CMD/ENTRYPOINT

System containers also do not need a CMD directive. The installation command atomic install --system checks out a local copy of the image and creates necessary files. To set up necessary environment/files/scripts, instead do so through runc's config.json.template or systemd tmpfiles (tmpfiles.template).

If the container can double as a docker container, then the CMD directive should follow the same guidelines as a docker container as highlighted above.

Other

A system container only requires a bare minimum of a Dockerfile, but for most services that is not enough to create a working service. For more detailed guidelines on specifications of other files, please visit Container:System_Container

systemd Containers

A systemd-based container runs systemd(/sbin/init) by default as its entrypoint. One or more services are configured using unit files, the say way they would be on a physical system or a virtual machine.

Benefits for systemd based containers

systemd based containers have some advantages over running a service directly as the entrypoint (pid 1):

• Support for multi service containers.
• Support for unit file mechanism for starting the container.
• Ability to reap zombie processes.
• Population of /tmp with content needed to be run by a service
• Proper handling of syslog messages
• Proper handling of journalctl messages

Issues with systemd based containers

STDOUT/STDERR of the container does not come back though the container runtime logging system. As a result, on a container host without oci-systemd-hook and oci-register-machine, log files will not appear on the host, since there’ll be no way to make the logs available outside of the container.

In addition, systemd adds some requirements on the mechanism that containers are run:

• systemd requires that a tmpfs be mounted at /run.
• Systemd inside the container wants container environment variable defined. (container=oci)
• systemd requires that /sys/fs/cgroup be available inside of the container
• systemd requires that the signal to stop the container is SIGRTMIN+3
• systemd likes if /tmp is a tmpfs.

For more information on systemd in a container read: https://developers.redhat.com/blog/2016/09/13/running-systemd-in-a-non-privileged-container/

LABELS

• run/usage: Installing the oci-systemd-hook and oci-register-machine packages alongside the docker packaged shipped in Fedora takes care of most of the setup for running a systemd container. However, in order to provide compatibility with other container hosts which do not include oci-systemd-hook and oci-register-machine, systemd-based containers should include a run or usage label that directs users to include the "--tmpfs /run", "--tmpfs /tmp", and "-v /sys/fs/cgroup:/sys/fs/cgroup" that systemd requires to run.

For example:

LABEL usage="docker run -d -P --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup SYSTEMDIMAGE"

CMD/ENTRYPOINT

systemd-based containers must include "/sbin/init" as an ENTRYPOINT or CMD, which will start systemd inside the container at run time. systemd will start up and manage services enabled with the Dockerfile using the standard "systemctl enable foo" commands in RUN statements.

For example:

FROM registry.fedoraproject.org/fedora:25
ENV container=oci
RUN dnf -y install httpd; dnf clean all; systemctl enable httpd
STOPSIGNAL SIGRTMIN+3
EXPOSE 80
CMD [ "/sbin/init" ]

In the future, Fedora will provide a base-init container called fedora-init. This image will be set up to run systemd containers by default. This means your Dockerfile could be as simple as

FROM registry.fedoraproject.org/fedora-init:25
RUN dnf -y install httpd; dnf clean all; systemctl enable httpd
EXPOSE 80

Content

Dockerfiles in Fedora should not contain net new code. The meaning of this is that software should be packaged properly as RPMs and placed in the Fedora repositories, Dockerfiles are simply a deliver mechanism for pre-defined "ready to run" configurations. This can be achieved as an Atomic App or similar. Any content that is to accompany the Dockerfile must either be configuration files or startup/orchestration scripts. The goal of this is such that we follow the key points of the Fedora Release Engineering Philosophy.

Multi Container Services

Each container image should provide only one "service" and multi-container services should be handled by an external orchestration tool at the users discretion such as OpenShift Origin, kubernetes, deis, Docker Swarm, Docker Compose, DC/OS, Cloud Foundry, Apache Mesos, etc.

These types of multi-container services should be documented in such a way that users can adapt them to their needs. One example would be using the Project Atomic nulecule specification.

Future
In the future there will be an user facing outlet such that this information can be index and searched so that it's easy for users to consume this content. Our long term Fedora Container Registry goals are still in development

Discussion

For suggestions, feedback, or to report issues with this page please contact the Fedora Cloud SIG.

See Talk:PackagingDrafts/Containers for discussion.

See also

Container:Review_Process