Koji/WritingKojiCode

This page gives an overview of the Koji code and then describes what needs to change if you want to add a new type of task. A new task could be for a new content type, or assembling the results of multiple builds together, or something else that helps your workflow. New contributors to Koji should leave this page knowing where to begin and have enough understanding of Koji's architecture to be able to estimate how much work is still ahead of them.

Component Overview

Koji is comprised of several components, this section goes into details for each one, and what you potentially may need to change. Every component is written in Python, so you will need to know that language beyond a beginner level.

Koji-Hub

koji-hub is the center of all Koji operations. It is an XML-RPC server running under mod_wsgi in Apache. koji-hub is passive in that it only receives XML-RPC calls and relies upon the build daemons and other components to initiate communication. koji-hub is the only component that has direct access to the database and is one of the two components that have write access to the file system. If you want to make changes to the webUI (new pages or themes), you are looking in the wrong section, there is a separate component for that.

Implementation Details

The hub/kojihub.py file is where the server-side code lives. If you need to fix any server problems or want to add any new tasks, you will need to modify this file. Changes to the database schema will almost certainly require code changes too. This file gets deployed to /usr/share/koji-hub/kojihub.py, whenever you make changes to that remember to restart httpd. Also there are cases where httpd looks for an existing .pyc file and takes it as-is, instead of re-compiling it when the code is changed.

In the code there are two large classes: RootExports and HostExports. RootExports exposes methods using XMLRPC for any client that connects to the server. The Koji CLI makes use of this quite a bit. If you want to expose a new API to any remote system, add your code here. The HostExports class does the same thing except it will ensure the requests are only coming from builders. Attempting to use an API exposed here with the CLI will fail. If your work requires the builders to call a new API, you should implement it here. Any other function defined in this file is inaccessible by remote hosts. It is generally a good practice to have the exposed APIs do very little work, and pass off control to internal functions to do the heavy lifting.

Database interactions are done with raw query strings, not with any kind of modern ORM. Consider using context objects from the Koji contexts library for thread-safe interactions. The database schema is captured in the docs directory in the root of a git clone. A visualization of the schema is not available at the time of this writing.

Troubleshooting

The hub runs in an Apache service, so you will need to look in Apache logs for error messages if you are encountering 500 errors or the service is failing to start. Specifically you want to check in:

• /var/log/httpd/error_log
• /var/log/httpd/ssl_error_log

If you need more specific tracebacks and debugging data, consider changing the debugging setting in /etc/koji-hub/hub.conf. Be advised the hub is very verbose with this setting on, your logs will take up gigabytes of space within several days.

Kojid

kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them accordingly. Essentially kojid asks koji-hub for work. Koji also has support for tasks other than building. Creating install images is one example. kojid is responsible for handling these tasks as well. kojid uses mock for building. It also creates a fresh buildroot for every build. kojid is written in Python and communicates with koji-hub via XML-RPC.

Implementation Details

The daemon runs as a service on a host that is traditionally not the same as the hub or webUI. This is a good security practice because the service runs as root, and executes untrusted code to produce builds on a regular basis. Keeping the Hub separate limits the damage a malicious package can do to the build system as a whole. For the same reason, the filesystem that the hub keeps built software on should be mounted Read-Only on the build host. It should call APIs on the hub that are exposed through the HostExports class in the hub code. Whenever the builder accepts a task, it forks a process to carry out the build.

An initscript/unit-file is available for kojid, so it can be stopped and started like a normal service. Remember to do this when you deploy changes!

TaskHandlers

All tasks in kojid have a TaskHandler class that defines what to do when the task is picked up from the hub. The base class is defined in koji/tasks.py where a lot of useful utility methods are available. An example is uploadFile, which is used to upload logs and built binaries from a completed build to the hub since the shared filesystem is read only.

The daemon code lives in builder/kojid, which is deployed to /usr/sbin/kojid. In there you'll notice that each task handler class has a Methods member and _taskWeight member. These must be defined, and the former is used to match the name of a waiting task (on the hub) with the task handler code to execute. Each task handler object must have a handler method defined, which is the entry point for the forked process when a builder accepts a task.

Tasks can have subtasks, which is a typical model when a build can be run on multiple architectures. In this case, developers should write 2 task handlers: one handles the build for exact one architecture, and one that assembles the results of those tasks into a single build, and sends status information to the hub. You can think of the latter handler as the parent task.

All task handler objects have a session object defined, which is the interface to use for communications with the hub. So, parent tasks should kick off child tasks using the session object's subtask method (which is part of HostExports). It should then call self.wait with all=True to wait for the results of the child tasks.

Here's a stub of what a new build task might look like:

class BuildThingTask(BaseTaskHandler):
  Methods = ['thing']
  _taskWeight = 0.5

  def handler(self, a, b, arches, options):
    subtasks = {}
    for arch in arches:
      subtasks[arch] = session.host.subtask(method='thingArch', a, b, arch)
    results = self.wait(subtasks.values(), all=True)
    # parse results and put rows in database
    # put files in their final resting place
    return 'Build successful'

class BuildThingArchTask(BaseTaskHandler):
  Methods = ['thingArch']
  _taskWeight = 2.0

  def handler(self, a, b, arch):
    # do the build, capture results in a variable
    self.uploadFile('/path/to/some/log')
    self.uploadFile('/path/to/binary/file')
    return result

Source Control Managers

If you your build needs to check out code from a Source Control Manager (SCM) such as git or subversion, you can use SCM objects defined in koji/daemon.py. They take a specially formed URL as an argument to the constructor. Here's an example use. The second line is important, it makes sure the SCM is in the whitelist of SCMs allowed in /etc/kojid/kojid.conf.

scm = SCM(url)
scm.assert_allowed(self.options.allowed_scms)
directory = scm.checkout('/checkout/path', session, uploaddir, logfile)

Checking out takes 4 arguments: where to checkout, a session object (which is how authentication is handled), a directory to upload the log to, and a string representing the log file name. Using this method Koji will checkout (or clone) a remote repository and upload a log of the standard output to the task results.

Build Root Objects

It is encouraged to build software in mock chroots if appropriate. That way Koji can easily track precise details about the environment in which the build was executed. In builder/kojid a BuildRoot class is defined, which provides an interface to execute mock commands. Here's an example of their use:

broot = BuildRoot(self.session, self.options, build_tag, arch, self.id)

A session object, task options, and a build tag should be passed in as-is. You should also specify the architecture and the task ID. If you ever need to pass in specialized options to mock, look in the ImageTask.makeImgBuildRoot method to see how they are defined and passed in to the BuildRoot constructor.

troubleshooting

The daemon writes a log file to /var/log/kojid.log. Debugging output can be turned on in /etc/kojid/kojid.conf.

Koji-Web

koji-web is a set of scripts that run in mod_wsgi and use the Cheetah templating engine to provide a web interface to Koji. It acts as a client to koji-hub providing a visual interface to perform a limited amount of administration. koji-web exposes a lot of information and also provides a means for certain operations, such as cancelling builds.

• cheetah
• index.py
• restarting apache
• displaying lists and dicts
• troubleshooting

Koji-client

koji-client is a command line interface that provides many hooks into Koji. It allows the user to query much of the data as well as perform actions such as adding users and initiating build requests.

• uploading work
• handler_ concept
• progress bars?
• make_task
• symlink trick for different configs
• fixing task args

Kojira

kojira is a daemon that keeps the build root repodata updated. It is responsible for removing redundant build roots and cleaning up after a build request is completed.

Building and Deploying Changes

• using the make file

Adding a New Task

Interested in adding a new task, or supporting some other type of build? This section aims to walk through what that will be like for a developer.

• overview of task flow
• per-component changes

Patch Review

If you have a patch to submit, please send it to buildsys-list@lists.fedoraproject.org. Some guidelines for a good patch are listed below too.

• guide 1
• guide 2