Latest revision as of 09:40, 13 September 2012

BuildSys Specification and Analysis

General Use Case

User logs in using OpenId.
User creates a #PrivateRepo. POLICY: who can create a repo, how many repos?
User alters the repo settings (#MockConfig, #Permissions, TODO: anything else?) as he wishes.
User uploads a number of SRPMs and lets them build with specified mock config(s). In next versions, these will also be submitted from sort-of-dist-git or by automatical builds from gems/pypi/cpan.
When the build is finished, it's results are transferred to the PrivateRepo's #RepoHome.
Everyone can download the repofile (probably as an installable RPM containing just the yum repofile) and install packages from it.

Builds In PrivateRepos

Summary of build process in a #PrivateRepo instance.

One or more SRPMs have been submitted to build with one or more #MockConfigs. This results in a parent #Task.
1. In future, the SRPMs may be also be built and submitted from "sort-of-dist" or provided by automatic conversion from Ruby Gems, etc.
BuildSys gets proper builders (proper system version, architecture) from a builder provider (cloud?).
BuildSys starts the builds (with mockchain/mockremote/...?) using specified #MockConfigs on the builders. This results into one or more sub#Tasks.
When the builds are finished (both successfully and unsuccessfully), results are transfered to the #RepoHome.
createrepo (=> new #Task) is run unless the builds were scratch or some of the builds failed.
The parent #Task ends here.

Terms, Definitions, Explanations

This section explains different terms and their meanings/functionality details.

PrivateRepo

The whole thing :)

The repo has multiple #MockConfigs, e.g. fedora(/centos?) release and architecture.
The repo has defined #Permissionss.

Permissions

POLICY: Each repo has only repo-level permissions. The permissions are be divided into these groups: repo-admin (specifies permissions of other users for the repo), repo-rel-eng (can add/modify/delete mock configs, trigger createrepo runs, delete old builds), repo-packager (can build packages, delete old builds). No package-level permissions are considered for first version.

MockConfig

A mock config inside a #PrivateRepo.

A single mock config in a repo (this should in fact be a full mock config file as in /etc/mock). POLICY: Using different base repos should be forbidden. The users should only be allowed to configure additional repos. The base repo will serve as a fixed point for determining architecture and OS version.
Each mock-config can be altered: adding more yum repos, altering minimal buildroot. To be prepared for sort-of-dist-git, we should have both SRPM build mock configs and RPM build mock configs. The first version should only use RPM build mock configs. When both are used (in later versions), they should come in pairs (e.g. each RPM build mock config should have an associated SRPM build mock config - or none, perhaps).

RepoHome

A machine serving as a storage for build results. Contains a yum repo, that holds packages for all the successful builds and logs (even for non-successful builds).

POLICY: how long will we store the packages/logs?

Task

Analogy of Koji task, e.g. an overall build task (parent of multiple build tasks)/build/createrepo run. Tasks can have subtasks.

Class Diagram

Available as image (http://bkabrda.fedorapeople.org/buildsys/buildsys-2.jpg) or DIA project (http://bkabrda.fedorapeople.org/buildsys/buildsys-2.dia).

Others

Things To Consider

Automatic builds of packages from rubygems/pypi/cpan - best approach probably be having an automatic checker (something like upstream release monitoring) that would monitor packages and create SRPMs, packagers would then just go through the automatically prepared SRPMs (adjust them) and let them build.
Using #PrivateRepos instead getting new set of tags in Koji - would probably require "sort-of-dist-git" to be able to merge the changes easily between Fedora and BuildSys.

Problems/More Policy Issues

Licensing of the RPM content
Tracking RPM origin - may turn out very important when having issues with content licensing
- RPMs should have their ?DB entry/special header? (don't know yet, that's why that is not in the class diagram) that would say:
- SRPM for binary RPM
- Origin info (who - userid; from where - SRPM upload or "sort-of-dist-git" or automatic build; some other info) for SRPM

@@ Line 9: / Line 9: @@
 # When the build is finished, it's results are transferred to the PrivateRepo's [[#RepoHome]].
 # Everyone can download the repofile (probably as an installable RPM containing just the yum repofile) and install packages from it.
 == Builds In PrivateRepos ==
 Summary of build process in a [[#PrivateRepo]] instance.
 # One or more SRPMs have been submitted to build with one or more [[#MockConfig]]s. This results in a parent [[#Task]].
+## In future, the SRPMs may be also be built and submitted from "sort-of-dist" or provided by automatic conversion from Ruby Gems, etc.
 # BuildSys gets proper builders (proper system version, architecture) from a builder provider (cloud?).
-# BuildSys starts the builds (with mockchain/mockremote/...?) using specified [[#MockConfig]]s on the builders. This results into one or more sub[[Task]]s.
+# BuildSys starts the builds (with mockchain/mockremote/...?) using specified [[#MockConfig]]s on the builders. This results into one or more sub[[#Task]]s.
 # When the builds are finished (both successfully and unsuccessfully), results are transfered to the [[#RepoHome]].
 # createrepo (=> new [[#Task]]) is run unless the builds were scratch or some of the builds failed.
@@ Line 34: / Line 34: @@
 A mock config inside a [[#PrivateRepo]].
 * A single mock config in a repo (this should in fact be a full mock config file as in /etc/mock). POLICY: Using different base repos should be forbidden. The users should only be allowed to configure additional repos. The base repo will serve as a fixed point for determining architecture and OS version.
-* Each mock-config can be altered: adding more yum repos, altering minimal buildroot. To be prepared for sort-of-dist-git, we should have both SRPM build mock configs and RPM build mock configs. The first version should only use RPM build mock configs. When both are used in later versions, they should come in pairs (e.g. each RPM build mock config should have either associated SRPM build mock config - or none, perhaps).
+* Each mock-config can be altered: adding more yum repos, altering minimal buildroot. To be prepared for sort-of-dist-git, we should have both SRPM build mock configs and RPM build mock configs. The first version should only use RPM build mock configs. When both are used (in later versions), they should come in pairs (e.g. each RPM build mock config should have an associated SRPM build mock config - or none, perhaps).
 === RepoHome ===
@@ Line 49: / Line 49: @@
 == Things To Consider ==
-* Automatic builds of packages from rubygems/pypi/cpan.
+* Automatic builds of packages from rubygems/pypi/cpan - best approach probably be having an automatic checker (something like upstream release monitoring) that would monitor packages and create SRPMs, packagers would then just go through the automatically prepared SRPMs (adjust them) and let them build.
-* Using [[#PrivateRepo]]s instead getting new set of tags in Koji.
+* Using [[#PrivateRepo]]s instead getting new set of tags in Koji - would probably require "sort-of-dist-git" to be able to merge the changes easily between Fedora and BuildSys.
 == Problems/More Policy Issues ==
 * Licensing of the RPM content
+* Tracking RPM origin - may turn out very important when having issues with content licensing
+** RPMs should have their ?DB entry/special header? (don't know yet, that's why that is not in the class diagram) that would say:
+** SRPM for binary RPM
+** Origin info (who - userid; from where - SRPM upload or "sort-of-dist-git" or automatic build; some other info) for SRPM

Search

User:Bkabrda/BuildSys: Difference between revisions