Introduction
This document specifies details of messages, which should be sent out by Upstream/Downstream CI systems to fedmsg/UMB on events related to artifacts testing. Currently there is low unification of these messages and there are various types of messages sent out by the CI systems. Unification will make it possible to build simpler microservices which will provide useful features for everyone.
Namespace
The messages will use a unique UMB virtual topic namespace to mitigate collisions with other systems:
- UMB: /topic/VirtualTopic.eng.ci
- fedmsg: org.fedoraproject.prod.ci
For all the tests on all the artifacts the systems will use this topic naming convention:
- UMB: /topic/VirtualTopic.eng.ci.<artifact>.test.{queued,running,complete,error}
- fedmsg: org.fedoraproject.prod.ci.<artifact>.test.{queued,running,complete,error}
where
- <artifact> - is one of the tested artifacts, see ARTIFACT special field definition, i.e. brew-build, redhat-container-image etc.
- queued (optional) - testing has been queued, but not yet started
- running (optional) - testing has been started and is in progress
- complete (mandatory) - testing has been completed
- error (mandatory) - testing has aborted, it was not finished (e.g. because of infrastructure error, CI system error, etc.)
Note that a test failure is not an error. Test failures should be exposed under the ci.test.complete topic.
Messages Anatomy
This section defines the contents of the messages without headers. All data should be stored in the body of the message. Additional fields can be added to the messages as required by the senders, although it is advised to document them here.
Syntax requirements for field names
- Use only characters [a-z0-9_]
- No spaces are allowed in the name of the field
- For separation of multi-word strings in field names use the underscore character
- Boolean value must be one of “true” or “false”
Requirements for field values
- If a field contains multiple values a list must be used for it
- e.g. recipients: [ “mvadkert”, “optak” ]
Special field values
ARTIFACT
Can be one of the defined build artifacts. This includes:
- brew-build - scratch or non-scratch build from https://brew.engineering.redhat.com
- fedora-update - a Fedora update (currently from Bodhi)
- ...
CI_NAME
A human readable name for the CI system, for example: “BaseOS CI”, “IdM CI”, “Libvirt CI”, “RPM factory”, etc.
CI_TEAM
A human readable name of the team running the testing. This is useful to distinguish multiple teams which running on the same Jenkins CI instance. For example: “BaseOS QE”, “RTT”, “Libvirt QE”, etc.
DOCKER_IMAGE_REPOSITORY
A docker repository url plus docker image identificator that is stable and unique. For example for this container image build it is:
somehost.redhat.com:8888/rhscl/varnish-4-rhel7:4-14
Note that other URLs might change in the future (e.g. latest, rhel-7:4, etc.). This information is available in the results[repositories] field of the original brew build task close message.
BUILD_TASK_SOURCE
The first item in the request field from task details. This is usually a link to git repository with a reference, delimited with the ‘#’ sign. For example (note that the commit has was truncated for brevity):
git+https://src.fedoraproject.org/rpms/rubygem-review.git?#5e0ae23a
In case of a scratch build or other build built via uploading a src.rpm the build task source will look like this example of a bash scratch build:
cli-build/1498396792.492652.jYJCrkUF/bash-4.4.12-5.fc26.src.rpm
BUILD_DEPENDENCIES
NVR’s of additional dependencies that were needed to install the brew/koji rpm build artifact. Note that usually to install a package, you might need to include packages, not yet in production repositories.
TEST_NAMESPACE
A namespace used to prefix the ResultsDB testcase name. This is usually something identifying your team. Examples: “osci”, “rtt”, “baseosci”, ...
TEST_STATUS
One of the following states (in conformance with ResultsDB 2.0 API valid outcome values):
- passed
- failed
- info
- needs_inspection
Last status is indicating failure, but needing manual review. Note that this is currently used only for RPMdiff testing.
TEST_CATEGORY
A list with these possible values:
- static-analysis
- functional
- integration
- validation
TEST_TYPE
A string. Identifies the test(s) being run in this execution. Depending on your test system this may indicate a single test (possibly with ‘scenario’ information as well), a group of tests, or you may only ever use one value here if your test system does not emit separate messages per test or test group. A good way to think of this is “the information a consumer would need to identify runs of ‘the same test(s)’ on two different artifacts.”
Examples::
- tier1
- tier2
- tier3
- rpmdiff-analysis
- rpmdiff-comparison
- covscan
- pipeline
- autocloud
- base_selinux KDE-live-iso x86_64 64bit
- ANY OTHER IDENTIFIER THAT MAKES SENSE FOR YOUR TEST SYSTEM
SYSTEM_PROVIDER
One of
- openstack
- beaker
- openshift
- rhev
- ANY_OTHER_PROVIDER
SYSTEM_OS_NAME
Identificator of the system, depending on the provider. In case of
- openstack - image name
- beaker - distro name
- openshift - container image name
- ANY_OTHER_NAME specific to the provider
SYSTEM_OS_ARCHITECTURE
Architecture of the system/distro used for testing. Examples: x86_64, ppc64le, s390x, aarch64
SYSTEM_OS_VARIANT
The compose or image variant, if applicable. Examples: Server, Workstation
GENERATED_AT
Time when the requested was generated. This can be useful to track various lags in the system of messages, e.g. datagrepper lagging in logging a message. This should be a string of UTC time in ISO format. In Python. You can generate the format via
import datetime
datetime.datetime.utcnow().isoformat(' ')
An example is:
2018-05-10 08:58:31.222602