|
|
| (95 intermediate revisions by 23 users not shown) |
| Line 1: |
Line 1: |
| | {{autolang|base=yes}} |
| '''Setting Up a Koji Build System''' | | '''Setting Up a Koji Build System''' |
|
| |
|
| The Koji components may ''live'' on separate resources as long as all resources are able to communicate. This document will cover how to setup each service individually, however, all services may ''live'' on the same resource.
| | This document has moved to the [https://docs.pagure.org/koji/ Koji Documentation]. The new location is: |
|
| |
|
| == PreReqs of Knowledge ==
| | https://docs.pagure.org/koji/server_howto/ |
|
| |
|
| * Basic understanding of SSL and authentication via certificates and/or Kerberos credentials
| | You can submit changes to Koji's docs in [https://pagure.io/koji Pagure] |
| * Basic knowledge about creating a database in PostgreSQL and importing a schema
| |
| * Working with psql
| |
| * Basic knowledge about Apache configuration
| |
| * Basic knowledge about yum/createrepo/mock - else you'll not be able to debug problems!
| |
| * Basic knowledge about using command line
| |
| * Basic knowledge about RPM building
| |
| * Simple usage of the Koji client
| |
| * For an overview of yum, mock, Koji (and all its subcomponents), mash, and how they all work together, see the excellent slides put together by Steve Traylen at CERN [http://indico.cern.ch/event/55091].
| |
| | |
| == PreReqs of Packages ==
| |
| | |
| '''On the server (koji-hub/koji-web)'''
| |
| * httpd
| |
| * mod_ssl
| |
| * postgresql-server
| |
| * mod_python (>= 3.3.1 for Kerberos authentication)
| |
| | |
| '''On the builder (koji-builder)'''
| |
| * mock
| |
| * setarch (for some archs you'll require a patched version)
| |
| * rpm-build
| |
| * createrepo
| |
| | |
| == A note on filesystem space ==
| |
| Koji will consume copious amounts of disk space under the primary KojiDir directory (as set in the kojihub.conf file). However, as koji makes use of mock on the backend to actually create build roots and perform the builds in those build roots, it might come to a surprise to users that a running koji server will consume large amounts of disk space under /var/lib/mock as well. Users should either plan the disk and filesystem allocations for this, or plan to modify the default mock build directory in the kojid.conf file.
| |
| | |
| == Koji Authentication Selection ==
| |
| Koji primarily supports Kerberos and SSL Certificate authentication. For basic koji command line access, plain user/pass combinations are possible. However, kojiweb does '''not''' support plain user/pass authentication. Furthermore, once either Kerberos or SSL Certificate authentication is enabled so that kojiweb will work, the plain user/pass method will stop working entirely. As such plain user/pass authentication is a stop gap measure at best unless you intend to never setup a fully functional kojiweb instance. We encourage skipping the plain user/pass method altogether and properly configuring either Kerberos or SSL Certification authentication from the start. Deciding on how to authenticate users colors all the other actions you take in setting up koji so it's a decision best made up front.
| |
| | |
| For Kerberos authentication, a working Kerberos environment (the user is assumed to either already have this or know how to set it up themselves, instructions for it are not included here) and the Kerberos credentials of the initial admin user will be necessary to bootstrap the user database.
| |
| | |
| For SSL authentication, SSL certificates for the xmlrpc server, for the various koji components, and one for the admin user will need to be setup (the user need not know how to create certificate chains already, we include the instructions for this below).
| |
| | |
| === Setting up SSL Certificates for authentication ===
| |
| | |
| '''Certificate generation'''
| |
| | |
| * Copy-and-paste the ssl.cnf listed here, save it in /etc/pki/koji/ssl.cnf, and then *edit* it! (0.organizationName_default should be modified at a minimum)
| |
| | |
| '''ssl.cnf'''
| |
| <pre>
| |
| HOME = .
| |
| RANDFILE = .rand
| |
| | |
| [ca]
| |
| default_ca = ca_default
| |
| | |
| [ca_default]
| |
| dir = .
| |
| certs = $dir/certs
| |
| crl_dir = $dir/crl
| |
| database = $dir/index.txt
| |
| new_certs_dir = $dir/newcerts
| |
| certificate = $dir/%s_ca_cert.pem
| |
| private_key = $dir/private/%s_ca_key.pem
| |
| serial = $dir/serial
| |
| crl = $dir/crl.pem
| |
| x509_extensions = usr_cert
| |
| name_opt = ca_default
| |
| cert_opt = ca_default
| |
| default_days = 3650
| |
| default_crl_days = 30
| |
| default_md = md5
| |
| preserve = no
| |
| policy = policy_match
| |
| | |
| [policy_match]
| |
| countryName = match
| |
| stateOrProvinceName = match
| |
| organizationName = match
| |
| organizationalUnitName = optional
| |
| commonName = supplied
| |
| emailAddress = optional
| |
| | |
| [req]
| |
| default_bits = 1024
| |
| default_keyfile = privkey.pem
| |
| distinguished_name = req_distinguished_name
| |
| attributes = req_attributes
| |
| x509_extensions = v3_ca # The extentions to add to the self signed cert
| |
| string_mask = MASK:0x2002
| |
| | |
| [req_distinguished_name]
| |
| countryName = Country Name (2 letter code)
| |
| countryName_default = AT
| |
| countryName_min = 2
| |
| countryName_max = 2
| |
| stateOrProvinceName = State or Province Name (full name)
| |
| stateOrProvinceName_default = Vienna
| |
| localityName = Locality Name (eg, city)
| |
| localityName_default = Vienna
| |
| 0.organizationName = Organization Name (eg, company)
| |
| 0.organizationName_default = My company
| |
| organizationalUnitName = Organizational Unit Name (eg, section)
| |
| commonName = Common Name (eg, your name or your server\'s hostname)
| |
| commonName_max = 64
| |
| emailAddress = Email Address
| |
| emailAddress_max = 64
| |
| | |
| [req_attributes]
| |
| challengePassword = A challenge password
| |
| challengePassword_min = 4
| |
| challengePassword_max = 20
| |
| unstructuredName = An optional company name
| |
| | |
| [usr_cert]
| |
| basicConstraints = CA:FALSE
| |
| nsComment = "OpenSSL Generated Certificate"
| |
| subjectKeyIdentifier = hash
| |
| authorityKeyIdentifier = keyid,issuer:always
| |
| | |
| [v3_ca]
| |
| subjectKeyIdentifier = hash
| |
| authorityKeyIdentifier = keyid:always,issuer:always
| |
| basicConstraints = CA:true
| |
| </pre>
| |
| | |
| '''Generate CA'''
| |
| The CA is the Certificate Authority. It's the key/cert pair used to sign all the other certificate requests. When configuring the various koji components, both the client CA and the server CA will be the CA generated here.
| |
| | |
| <pre>
| |
| cd /etc/pki/koji/
| |
| mkdir {certs,private}
| |
| touch index.txt
| |
| echo 01 > serial
| |
| caname=koji
| |
| openssl genrsa -out private/${caname}_ca_cert.key 2048
| |
| openssl req -config ssl.cnf -new -x509 -days 3650 -key private/${caname}_ca_cert.key \
| |
| -out ${caname}_ca_cert.crt -extensions v3_ca
| |
| </pre>
| |
| | |
| The last command above will ask you to confirm a number of items about the certificate you are generating. Presumably you already edited sane defaults for the country, state/province, locale, and organization into the ssl.cnf file and you only needed to hit enter. It's the organizational unit and the common name that we will be changing in the various certs we create. For the CA itself, these fields don't have a hard requirement.
| |
| | |
| '''Generate the koji component certificates and the admin certificate'''
| |
| | |
| Each koji component needs its own certificate to identify it. Two of the certificates (kojihub and kojiweb) are used as server side certificates that authenticate the server to the client. For this reason, you want the common name on both of those certs to be the fully qualified domain name of the web server they are running on so that clients don't complain about the common name and the server not being the same. For the other certificates (kojira, kojid, the initial admin account, and all user certificates), the cert is used to authenticate the client to the server and the common name in the cert is the login name. Therefore the common name must be same as the user name in the koji database. When using ''koji add-host'' to add a build machine into the koji database, it creates a user account for that host even though the user account doesn't appear in the user list. The user account created during an add-host operation must match the common name of the certificate used by the kojid program on that host. When creating the kojiweb certificate, you'll want to remember exactly what values you enter for each field as you'll have to regurgitate those into the /etc/koji-hub/hub.conf file as the ProxyDNs entry.
| |
| | |
| The loop below should help facilitate the creation of multiple certifications. Adjust the number of kojibuilders and the name of the admin account as you see fit. For much of this guide, the admin account is called "kojiadmin".
| |
| <pre>
| |
| for user in kojira kojiweb kojihub kojibuilder{1..5} admin-account; do
| |
| openssl genrsa -out certs/${user}.key 2048
| |
| openssl req -config ssl.cnf -new -nodes -out certs/${user}.csr -key certs/${user}.key
| |
| openssl ca -config ssl.cnf -keyfile private/${caname}_ca_cert.key -cert ${caname}_ca_cert.crt \
| |
| -out certs/${user}.crt -outdir certs -infiles certs/${user}.csr
| |
| cat certs/${user}.crt certs/${user}.key > ${user}.pem
| |
| done
| |
| </pre>
| |
| | |
| ''' Generate a PKCS12 user certificate (for web browser)'''
| |
| This is only required for user certificates.
| |
| <pre>
| |
| openssl pkcs12 -export -inkey certs/${user}.key -in certs/${user}.crt -CAfile ${caname}_ca_cert.crt \
| |
| -out certs/${user}_browser_cert.p12
| |
| </pre>
| |
| | |
| When generating certs for a user, the user will need the ${user}.pem, the ${caname}_ca_cert.crt, and the ${user}_browser_cert.p12 files. The ${user}.pem file would normally be installed as ~/.fedora.cert, the ${caname}_ca_cert.crt file would be installed as both ~/.fedora-upload-ca.cert and ~/.fedora-server-ca.cert, and the user would import the ${user}_brower_cert.p12 into their web browser as a personal certificate. The user must specify <code>authtype = ssl</code> under <code>[koji]</code> in the configuration for the command-line koji client.
| |
| | |
| '''Copy certificates into ~/.koji for kojiadmin'''
| |
| | |
| You're going to want to be able to send admin commands to the kojihub. In order to do so, you'll need to use the newly created certificates to authenticate with the hub. Copy the certificates for the koji CA and the kojiadmin user to ~/.koji:
| |
| <pre>
| |
| kojiadmin@localhost$ mkdir ~/.koji
| |
| kojiadmin@localhost$ cp /etc/pki/koji/kojiadmin.pem ~/.koji/client.crt
| |
| kojiadmin@localhost$ cp /etc/pki/koji/koji_ca_cert.crt ~/.koji/clientca.crt
| |
| kojiadmin@localhost$ cp /etc/pki/koji/koji_ca_cert.crt ~/.koji/serverca.crt
| |
| </pre>
| |
| | |
| '''Note:''' See /etc/koji.conf for the current system wide koji client configuration. Copy /etc/koji.conf to ~/.koji/config if you wish to change the config on a per user basis.
| |
| | |
| === Setting up Kerberos for authentication ===
| |
| The initial configuration of a kerberos service is outside the scope of this document, however there are a few specific things required by koji.
| |
| | |
| '''DNS'''
| |
| | |
| The koji builders (kojid) use DNS to find the kerberos servers for any given realm.
| |
| | |
| <pre>
| |
| _kerberos._udp IN SRV 10 100 88 kerberos.EXAMPLE.COM.
| |
| </pre>
| |
| The trailing dot denotes DNS root and is needed if FQDN is used.
| |
| | |
| | |
| '''Principals and Keytabs'''
| |
| | |
| It should be noted that in general you will need to use the fully qualified domain name of the hosts when generating the keytabs for services.
| |
| | |
| You will need the following principals extracted to a keytab for a fully kerborised configuration, the requirement for a host key for the koji-hub is currently hard coded into the koji client.
| |
| | |
| * host/kojihub@EXAMPLE.COM
| |
| ** Used by the koji-hub server when communicating with the koji client
| |
| * HTTP/kojiweb@EXAMPLE.COM
| |
| ** Used by the koji-web server when performing a negotiated Kerberos authentication with a web browser. This is a service principal for Apache's mod_auth_kerb.
| |
| * koji/kojiweb@EXAMPLE.COM
| |
| ** Used by the koji-web server during communications with the koji-hub. This is a user principal that will authenticate koji-web to Kerberos as "koji/kojiweb@EXAMPLE.COM". Koji-web will proxy the mod_auth_kerb user information to koji-hub (the <code>ProxyPrincipals</code> koji-hub config option).
| |
| * koji/kojira@EXAMPLE.COM
| |
| ** Used by the kojira server during communications with the koji-hub
| |
| * compile/builder1@EXAMPLE.COM
| |
| ** Used on builder1 to communicate with the koji-hub
| |
| | |
| == PostgreSQL Server ==
| |
| The PostgreSQL server must be installed and the database primed.
| |
| | |
| '''Configuration Files:'''
| |
| | |
| * /var/lib/pgsql/data/pg_hba.conf
| |
| * /var/lib/pgsql/data/postgresql.conf
| |
| | |
| '''Install PostgreSQL:'''
| |
| <pre>
| |
| root@localhost$ yum install postgresql-server
| |
| </pre>
| |
| | |
| '''Initialize PostgreSQL DB:'''
| |
| <pre>
| |
| root@localhost$ service postgresql initdb
| |
| root@localhost$ service postgresql start
| |
| </pre>
| |
| | |
| '''Setup User Accounts:'''
| |
| <pre>
| |
| root@localhost$ useradd koji
| |
| root@localhost$ passwd -d koji
| |
| </pre>
| |
| | |
| '''Setup PostgreSQL and populate schema:'''
| |
| | |
| <pre>
| |
| root@localhost$ su - postgres
| |
| postgres@localhost$ createuser koji
| |
| Shall the new role be a superuser? (y/n) n
| |
| Shall the new role be allowed to create databases? (y/n) n
| |
| Shall the new role be allowed to create more new roles? (y/n) n
| |
| postgres@localhost$ createdb -O koji koji
| |
| postgres@localhost$ logout
| |
| root@localhost$ su - koji
| |
| koji@localhost$ psql koji koji < /usr/share/doc/koji*/docs/schema.sql
| |
| koji@localhost$ exit
| |
| </pre>
| |
| | |
| '''Authorize Koji-web and Koji-hub resources:'''
| |
| ''In this example, Koji-web and Koji-hub are running on localhost.''
| |
| | |
| /var/lib/pgsql/data/pg_hba.conf:
| |
| ''These settings need to be valid and inline with other services configurations.''
| |
| ''Please note, the first matching auth line is used so this line must be above any other potential matches.''
| |
| ''Add:''
| |
| <pre>
| |
| host koji koji 127.0.0.1/32 trust
| |
| </pre>
| |
| ''You can also use UNIX socket access. The DBHost variable must be unset to use this method.''
| |
| ''Add:''
| |
| <pre>
| |
| local koji apache trust
| |
| local koji koji trust
| |
| </pre>
| |
| | |
| '''Make auth changes live:'''
| |
| <pre>
| |
| root@localhost$ su - postgres
| |
| postgres@localhost$ pg_ctl reload
| |
| postgres@localhost$ exit
| |
| </pre>
| |
| | |
| '''Bootstrapping the initial koji admin user into the PostgreSQL database:'''
| |
| The initial admin user must be manually added to the user database using sql commands. Once they are added and given admin privilege, they may add additional users and change privileges of those users via the koji command line tool's administrative commands. However, if you choose to use the simple user/pass method of authentication, then any password setting/changing must be done manually via sql commands as there is no password manipulation support exposed through the koji tools.
| |
| | |
| The sql commands you need to use vary by authentication mechanism.
| |
| | |
| '''User/Password Authentication:'''
| |
| <pre>
| |
| koji@localhost$ psql
| |
| koji=> insert into users (name, password, status, usertype) values ('admin-user-name', 'admin-password-in-plain-text', 0, 0);
| |
| </pre>
| |
| | |
| '''Kerberos authentication:''' The process is very similar to user/pass except you would replace the first insert above with this:
| |
| | |
| <pre>
| |
| koji=> insert into users (name, krb_principal, status, usertype) values ('admin-user-name', 'admin@EXAMPLE.COM', 0, 0);
| |
| </pre>
| |
| | |
| '''SSL Certificate authentication:''' there is no need for either a password or a Kerberos principal, so this will suffice:
| |
| | |
| <pre>
| |
| koji=> insert into users (name, status, usertype) values ('admin-user-name', 0, 0);
| |
| </pre>
| |
| | |
| '''Give yourself admin permissions'''
| |
| <pre>
| |
| koji=> insert into user_perms (user_id, perm_id, creator_id) values (<id of user inserted above>, 1, <id of user inserted above>);
| |
| </pre>
| |
| | |
| Note: you can get the ID of the new user by running the query:
| |
| <pre>
| |
| koji=> select * from users;
| |
| </pre>
| |
| | |
| You can't actually log in and perform any actions until kojihub is up and running in your web server. In order to get to that point you still need to complete the authentication setup and the kojihub configuration. If you wish to access koji via a web browser, you will also need to get kojiweb up and running.
| |
| | |
| == Koji Hub ==
| |
| | |
| Koji-hub is the center of all Koji operations. It is an XML-RPC server running under mod_python in Apache. koji-hub is passive in that it only receives XML-RPC calls and relies upon the build daemons and other components to initiate communication. Koji-hub is the only component that has direct access to the database and is one of the two components that have write access to the file system.
| |
| | |
| '''Configuration Files:'''
| |
| | |
| * /etc/httpd/conf/httpd.conf
| |
| * /etc/httpd/conf.d/kojihub.conf
| |
| * /etc/httpd/conf.d/ssl.conf (when using ssl auth)
| |
| * /etc/koji-hub/hub.conf
| |
| | |
| '''Install koji-hub:'''
| |
| <pre>
| |
| root@localhost$ yum install koji-hub httpd mod_ssl mod_python
| |
| </pre>
| |
| | |
| ==== Required Configuration ====
| |
| | |
| /etc/httpd/conf/httpd.conf:
| |
| The apache web server has two places that it sets maximum requests a server will handle before the server restarts. The xmlrpc interface in kojihub is a python application, and mod_python can sometimes grow outrageously large when it doesn't reap memory often enough. As a result, if you let apache handle infinite requests without respawning a new server, you can take a box down under heavy load. It's therefore strongly suggested that you set both instances of MaxRequestsPerChild in httpd.conf to something reasonable (at 100 the httpd processes will grow to about 75MB resident set size before respawning).
| |
| | |
| <pre>
| |
| <IfModule prefork.c>
| |
| ...
| |
| MaxRequestsPerChild 100
| |
| </IfModule>
| |
| <IfModule worker.c>
| |
| ...
| |
| MaxRequestsPerChild 100
| |
| </IfModule>
| |
| </pre>
| |
| | |
| /etc/koji-hub/hub.conf:
| |
| ''These settings need to be valid and inline with other services configurations.''
| |
| <pre>
| |
| DBName = koji
| |
| DBUser = koji
| |
| DBHost = db.example.com
| |
| KojiDir = /mnt/koji
| |
| LoginCreatesUser = On
| |
| KojiWebURL = http://kojiweb.example.com/koji
| |
| </pre>
| |
| | |
| ==== Optional Configuration ====
| |
| | |
| /etc/koji-hub/hub.conf:
| |
| ''If using Kerberos, these settings need to be valid and inline with other services configurations.''
| |
| <pre>
| |
| AuthPrincipal host/kojihub@EXAMPLE.COM
| |
| AuthKeytab /etc/koji.keytab
| |
| ProxyPrincipals koji/kojiweb@EXAMPLE.COM
| |
| HostPrincipalFormat compile/%s@EXAMPLE.COM
| |
| </pre>
| |
| | |
| /etc/koji-hub/hub.conf:
| |
| ''If using SSL auth, these settings need to be valid and inline with other services configurations for kojiweb to allow logins.
| |
| ProxyDNs should be set to the DN of the kojiweb certificate.''
| |
| <pre>
| |
| DNUsernameComponent = CN
| |
| ProxyDNs = "/C=US/ST=Massachusetts/O=Example Org/OU=Example User/CN=example/emailAddress=example@example.com"
| |
| </pre>
| |
| | |
| /etc/httpd/conf.d/kojihub.conf:
| |
| ''If using SSL auth, uncomment these lines for kojiweb to allow logins."
| |
| <pre>
| |
| <Location /kojihub>
| |
| SSLOptions +StdEnvVars
| |
| </Location>
| |
| </pre>
| |
| | |
| /etc/httpd/conf.d/ssl.conf:
| |
| ''Add the needed SSL options for apache.''
| |
| <pre>
| |
| SSLCertificateFile /etc/pki/koji/certs/kojihub.crt
| |
| SSLCertificateKeyFile /etc/pki/koji/certs/kojihub.key
| |
| SSLCertificateChainFile /etc/pki/koji/koji_ca_cert.crt
| |
| SSLCACertificateFile /etc/pki/koji/koji_ca_cert.crt
| |
| SSLVerifyClient require
| |
| SSLVerifyDepth 10
| |
| </pre>
| |
| | |
| ==== SELinux Configuration ====
| |
| If running in Enforcing mode, you will need to allow apache to connect to the postgreSQL server.
| |
| | |
| <pre>
| |
| root@localhost$ setsebool -P httpd_can_network_connect_db 1
| |
| </pre>
| |
| | |
| ==== Koji filesystem skeleton ====
| |
| | |
| Above in the kojihub.conf file we set KojiDir to /mnt/koji. For certain reasons, if you change this, you should make a symlink from /mnt/koji to the new location (note: this is a bug and should be fixed eventually). However, before other parts of koji will operate properly, we need to create a skeleton filesystem structure for koji as well as make the file area owned by apache so that the xmlrpc interface can write to it as needed.
| |
| | |
| <pre>
| |
| cd /mnt
| |
| mkdir koji
| |
| cd koji
| |
| mkdir {packages,repos,work,scratch}
| |
| chown apache.apache *
| |
| </pre>
| |
| | |
| At this point, you can now restart the web server and you should have at least minimal operation. The admin user should be able to connect via the command line client, add new users, etc. It's possible at this time to undertake initial administrative steps such as adding users and hosts to the koji database. For example, the kojira component needs repo privileges, but if you just let the account get auto created the first time you run kojira, it won't have that privilege, so you could pre-create the account and grant it the repo privilege now.
| |
| | |
| Ensure that your client is configured to work with your server. The system-wide koji client configuration file is ''/etc/koji.conf'', and the user-specific one is in ''~/.koji/config''. You may also use the "-c" option when using the Koji client to specify an alternative configuration file. If you are using SSL for authentication, specify <code>authtype = ssl</code> under <code>[koji]</code> in your Koji client configuration.
| |
| | |
| <pre>
| |
| kojiadmin@localhost$ koji add-user kojira
| |
| kojiadmin@localhost$ koji grant-permission repo kojira
| |
| </pre>
| |
| | |
| For similar technical reasons, you need to add-host each build host prior to starting kojid on that host the first time and could also do that now.
| |
| | |
| <pre>
| |
| kojiadmin@localhost$ koji add-host kojibuilder1 x86_64 i386
| |
| kojiadmin@localhost$ koji add-host kojibuilder2 ppc ppc64
| |
| kojiadmin@localhost$ koji add-host kojibuilder3 ia64
| |
| </pre>
| |
| | |
| == Koji Web - Interface for the Masses ==
| |
| | |
| Koji-web is a set of scripts that run in mod_python and use the Cheetah templating engine to provide an web interface to Koji. koji-web exposes a lot of information and also provides a means for certain operations, such as cancelling builds.
| |
| | |
| '''Configuration Files:'''
| |
| | |
| * /etc/httpd/conf.d/kojiweb.conf
| |
| * /etc/httpd/conf.d/ssl.conf
| |
| | |
| '''Install Koji-Web:'''
| |
| <pre>
| |
| root@localhost$ yum install koji-web mod_ssl
| |
| </pre>
| |
| | |
| ==== Required Configuration ====
| |
| /etc/httpd/conf.d/kojiweb.conf:
| |
| <pre>
| |
| PythonOption KojiHubURL http://hub.example.com/kojihub
| |
| PythonOption KojiWebURL http://www.example.com/koji
| |
| PythonOption KojiPackagesURL http://server.example.com/mnt/koji/packages
| |
| | |
| | |
| PythonOption WebCert /etc/pki/koji/kojiweb.pem
| |
| PythonOption ClientCA /etc/pki/koji/koji_ca_cert.crt
| |
| PythonOption KojiHubCA /etc/pki/koji/koji_ca_cert.crt
| |
| | |
| PythonOption LoginTimeout 72
| |
| PythonOption Secret CHANGE_ME
| |
| </pre>
| |
| | |
| ==== Optional Configuration ====
| |
| | |
| /etc/httpd/conf.d/kojiweb.conf:
| |
| ''If using Kerberos, these settings need to be valid and inline with other services configurations.''
| |
| <pre>
| |
| <Location /koji/login>
| |
| AuthType Kerberos
| |
| AuthName "Koji Web UI"
| |
| KrbMethodNegotiate on
| |
| KrbMethodK5Passwd off
| |
| KrbServiceName HTTP
| |
| KrbAuthRealm EXAMPLE.COM
| |
| Krb5Keytab /etc/httpd.keytab
| |
| KrbSaveCredentials off
| |
| Require valid-user
| |
| ErrorDocument 401 /koji-static/errors/unauthorized.html
| |
| </Location>
| |
| </pre>
| |
| | |
| /etc/httpd/conf.d/kojiweb.conf:
| |
| ''If using SSL auth, these settings need to be valid and inline with other services configurations.''
| |
| <pre>
| |
| <Location /koji/login>
| |
| SSLOptions +StdEnvVars
| |
| </Location>
| |
| </pre>
| |
| | |
| /etc/httpd/conf.d/ssl.conf:
| |
| ''Add the needed SSL options for apache.''
| |
| <pre>
| |
| SSLVerifyClient require
| |
| SSLVerifyDepth 10
| |
| </pre>
| |
| | |
| If you are using SSL authentication, the "PythonOption WebCert" file above must contain both the public ''and'' private key.
| |
| | |
| ==== Web interface now operational ====
| |
| | |
| At this point you should be able to point your web browser at the kojiweb URL and be presented with the koji interface. Many operations should work in read only mode at this point, and any configured users should be able to log in.
| |
| | |
| == Koji Daemon - Builder ==
| |
| | |
| Kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them accordingly. Koji also has support for tasks other than building such as creating livecd images or raw disk images, and kojid is responsible for handling these tasks as well. kojid uses mock for creating pristine build environments and creates a fresh one for every build, ensuring that artifacts of build processes cannot contaminate each other. kojid is written in Python and communicates with koji-hub via XML-RPC.
| |
| | |
| '''Configuration Files:'''
| |
| | |
| * /etc/kojid/kojid.conf - Koji Daemon Configuration
| |
| * /etc/sysconfig/kojid - Koji Daemon Switches
| |
| | |
| '''Install kojid:'''
| |
| <pre>
| |
| root@localhost$ yum install koji-builder
| |
| </pre>
| |
| | |
| ==== Required Configuration ====
| |
| | |
| /etc/kojid/kojid.conf:
| |
| ''This needs to point at your koji-hub.''
| |
| <pre>
| |
| ; The URL for the xmlrpc server
| |
| server=http://hub.example.com/kojihub
| |
| | |
| ; the username has to be the same as what you used with add-host
| |
| ; in this example follow as below
| |
| user = kojibuilder1.example.com
| |
| </pre>
| |
| ''This item may be changed, but may not be the same as KojiDir on the kojihub.conf file (although it can be something under KojiDir, just not the same as KojiDir)''
| |
| <pre>
| |
| ; The directory root for temporary storage
| |
| workdir=/tmp/koji
| |
| </pre>
| |
| | |
| ==== Optional Configuration (SSL certificates) ====
| |
| | |
| /etc/kojid/kojid.conf:
| |
| ''If using SSL, these settings need to be valid.''
| |
| <pre>
| |
| ;client certificate
| |
| ; This should reference the builder certificate we created above, for
| |
| ; kojibuilder1.example.com
| |
| cert = /etc/kojid/kojid.pem
| |
| | |
| ;certificate of the CA that issued the client certificate
| |
| ca = /etc/kojid/koji_ca_cert.crt
| |
| | |
| ;certificate of the CA that issued the HTTP server certificate
| |
| serverca = /etc/kojid/koji_ca_cert.crt
| |
| </pre>
| |
| | |
| ==== Optional Configuration (Kerberos Authentication) ====
| |
| | |
| /etc/kojid/kojid.conf:
| |
| ''If using Kerberos, these settings need to be valid and inline with other services configurations.''
| |
| | |
| <pre>
| |
| ; the username has to be the same as what you used with add-host
| |
| ;user =
| |
| | |
| host_principal_format=compile/%s@EXAMPLE.COM
| |
| </pre>
| |
| | |
| By default it will look for the Kerberos keytab in /etc/kojid/kojid.keytab
| |
| | |
| '''Note:''' Kojid will not attempt kerberos authentication to the koji-hub unless the username field is commented out
| |
| | |
| ==== Add the host entry for the koji builder to the database ====
| |
| Make sure you do this before you start kojid for the first time,
| |
| or you'll need to manually remove entries from the sessions and users
| |
| table before it can be run successfully.
| |
| <pre>
| |
| kojiadmin@localhost$ koji add-host kojibuilder1.example.com i386 x86_64
| |
| </pre>
| |
| | |
| ==== Add the host to the createrepo channel ====
| |
| Channels are a way to control which builders process which tasks. By default
| |
| hosts are added to the ''default'' channel. At least some build hosts also needs to be added
| |
| to the ''createrepo'' channel so there will be someone to process repo creation tasks initiated by kojira.
| |
| | |
| <pre>
| |
| kojiadmin@localhost$ koji add-host-to-channel kojibuilder1.example.com createrepo
| |
| </pre>
| |
| | |
| ==== A note on capacity ====
| |
| The default capacity of a host added to the host database is 2. This means that once the load average on that machine exceeds 2, kojid will not accept any additional tasks. This is separate from the maxjobs item in the configuration file. Before kojid will accept a job, it must pass both the test to ensure the load average is below capacity and that the current number of jobs it is already processing is less than maxjobs. However, in today's modern age of quad core and higher CPUs, a load average of 2 is generally insufficient to fully utilize hardware. As there is not an option to set the capacity of the host via the command line tools, it must be done manually in psql.
| |
| | |
| <pre>
| |
| koji@localhost$ psql koji
| |
| koji=# select (id, name, capacity) from host;
| |
| row
| |
| --------------------------------
| |
| (1,kojibuilder1.example.com,2)
| |
| (2,kojibuilder2.example.com,2)
| |
| (2 rows)
| |
| | |
| koji=# update host set capacity = 16 where id = 1;
| |
| UPDATE 1
| |
| koji=#
| |
| </pre>
| |
| | |
| ==== Start Kojid ====
| |
| <pre>
| |
| root@localhost$ /sbin/service kojid start
| |
| </pre>
| |
| Check /var/log/kojid.log to verify that kojid has started successfully.
| |
| | |
| == Kojira - Yum repository creation and maintenance ==
| |
| '''Configuration Files:'''
| |
| | |
| * /etc/kojira/kojira.conf - Kojira Daemon Configuration
| |
| * /etc/sysconfig/kojira - Kojira Daemon Switches
| |
| | |
| '''Install kojira'''
| |
| <pre>
| |
| root@localhost$ yum install koji-utils
| |
| </pre>
| |
| | |
| ==== Required Configuration ====
| |
| | |
| /etc/kojira/kojira.conf:
| |
| ''This needs to point at your koji-hub.''
| |
| <pre>
| |
| ; The URL for the xmlrpc server
| |
| server=http://hub.example.com/kojihub
| |
| </pre>
| |
| | |
| <br>
| |
| | |
| '''Additional Notes:'''
| |
| * Kojira needs read-write access to /mnt/koji.
| |
| * There should only be one instance of kojira running at any given time.
| |
| * It is not recommended that kojira run on the builders, as builders only should require read-only access to /mnt/koji.
| |
| | |
| ==== Optional Configuration ====
| |
| | |
| /etc/kojira/kojira.conf:
| |
| ''If using SSL, these settings need to be valid.''
| |
| <pre>
| |
| ;client certificate
| |
| ; This should reference the kojira certificate we created above
| |
| cert = /etc/pki/koji/kojira.pem
| |
| | |
| ;certificate of the CA that issued the client certificate
| |
| ca = /etc/pki/koji/koji_ca_cert.crt
| |
| | |
| ;certificate of the CA that issued the HTTP server certificate
| |
| serverca = /etc/pki/koji/koji_ca_cert.crt
| |
| </pre>
| |
| | |
| ''If using Kerberos, these settings need to be valid.''
| |
| <pre>
| |
| ; For Kerberos authentication
| |
| ; the principal to connect with
| |
| principal=koji/kojira@EXAMPLE.COM
| |
| ; The location of the keytab for the principal above
| |
| keytab=/etc/kojira.keytab
| |
| </pre>
| |
| | |
| /etc/sysconfig/kojira:
| |
| ''The local user kojira runs as needs to be able to read and write to /mnt/koji/repos/.
| |
| If the volume that directory resides on is root-squashed or otherwise unmodifiable
| |
| by root, you can set '''RUNAS=''' to a user that has the required privileges.''
| |
| | |
| ==== Add the user entry for the kojira user ====
| |
| If you did not already do so above, create the kojira user, and grant it the '''repo''' permission.
| |
| <pre>
| |
| kojiadmin@localhost$ koji add-user kojira
| |
| kojiadmin@localhost$ koji grant-permission repo kojira
| |
| </pre>
| |
| | |
| === Start Kojira ===
| |
| <pre>
| |
| root@localhost$ /sbin/service kojira start
| |
| </pre>
| |
| Check /var/log/kojira/kojira.log to verify that kojira has started successfully.
| |
| | |
| == Bootstrapping the Koji build environment ==
| |
| For instructions on importing packages and preparing Koji to run builds, see [[Koji/ServerBootstrap| Server Bootstrap]] .
| |
| | |
| For instructions on using External Repos and preparing Koji to run builds, see [[Koji/ExternalRepoServerBootstrap| External Repo Server Bootstrap]] .
| |
| | |
| Useful scripts and config files for setting up a Koji instance are available [http://fedora.danny.cz/koji/ here]
| |
| | |
| == Minutia and Miscellany ==
| |
| Please see [[Koji/KojiMisc| KojiMisc]] for additional details and notes about operating a koji server.
| |
