User:Hhorak/Draft/centos-scl-git-proposal

This is a proposal for SCL dist-git structure on CentOS, i.e git repositories where future upstream development of Software Collections will take place. It focuses on easy collaboration of Red Hat engineers, that will be able to merge changes from CentOS into internal dist-git or cherry-pick particular commits as easy as possible.

SCL SIG has been established and approved by CentOS board for driving this effort: http://wiki.centos.org/SpecialInterestGroup/SCLo

Synchronized content of RHSCL being done so far

Currently, source RPMs of all released RHSCL packages are synchronized to git.centos.org as well, but these are not subject of this proposal. They are imported into git.centos.org under rpms project and use SCL prefix in their component name.

Components vs. git repositories

Similar to Fedora, Red Hat uses one git repository for one component internally. It allows to set permissions with enough granularity.

Only members of SCL SIG will be able to commit to the dist-git, it is a requirement from [policy]

Structure of the repository will be the same as it is used in Fedora and Red Hat internally. It means every repository will contain at least RPM specfile and sources file which includes list of links to look-aside cache (with checksum). Optionally, there may be patches and other sources directly in git repository's root. No directories are used.

Branches

When working with Software Collections it is common that one package is part of more software collections. A typical example is some python module, that is available in python27 and python33 software collection as well. Changes between RPM sources for python27 and python33 are typically small or even none.

On the other hand, even though in some cases the same source is used in all branches, dist-git needs to be flexible and make it easy for maintainers to include different code for different collections. Typical use case will be Ruby on Rails (RoR) collection, where we find different versions of components in various RoR versions. Requiring the same sources for those packages in all branches would result in complicated and ugly spec files with many %ifs, which is not desired.

A similar scenario works for sources for different platforms -- CentOS 6 and CentOS 7. Thus, it make sense to have a separate branches for specific CentOS versions as well.

To summarize it, we will have one branch for each supported combination of CentOS version and SCL, where a component is build for. For example, we will have these branches (for packages that are available in python27 and python33 collection):

• scl-python27-el6
• scl-python27-el7
• scl-python33-el6
• scl-python33-el7

Prefix scl is used to easily identify that a branch is related to Software collections.

Repositories naming

Since we use branches to distinguish source code for particular CentOS version and SCL, no prefix is used in git repository name.

For example, dist-git will include the following git repositories:

• python
• python-meta
• python-setuptools

Git repositories for meta packages components will be called differently than it is done internally in Red Hat. Meta packages will use '-meta' suffix after the name of the collection without version. Using this layout it is easier to maintain meta packages for similar collections.

Content of repositories

The most often reason for having SCLs on CentOS is to provide newer versions of some component. Thus, content of SCLs in dist-git will often correspond with content in a stable Fedora. The difference will be in SCL-related macros added, which is not allowed in Fedora currently.

Naming of built packages

When built in koji, names of resulting RPM packages need to stay the same as packages in RHSCL, so they are compatible between each other. This is required by SCL consumers who do not want to differ between SCLs provided by CentOS and Red Hat.

From the same reason, SCLs will use rh as a vendor, which is used in RHSCL.

This is an example of packages that will be built in koji for particular branch:

component	sclo-python33-el6	sclo-python34-el6
python-meta	python33	rh-python34 *
python	python33-python	rh-python34-python *
python-setuptools	python33-python-setuptools	rh-python34-python-setuptools *
python-pytest	python33-python-pytest	rh-python34-python-pytest *

* Note about rh- prefix in the name
The prefix rh- is a planned feature for new collections so they won't be mixed with non-SCL packages called the same (e.g. mysql55 packaged usually may be mixed up with meta package for mysql55 collection. Already released SCLs will not use vendor in the packages names in order to stay compatible.

koji setup

koji setup for every branch includes setting up:

• a target/tag for every branch
• buildroot that includes the following packages:
  • scl-utils-build
  • colname00-build

Members of SCL SIG will be able to setup the koji: buildroots, tags, targets.

CentOS look-aside cache

Sources that are not part of git are uploaded into CentOS look-aside cache, similar to what is in Fedora.

Contributors will be able to upload any sources into CentOS look-aside cache by a tool provided by CentOS (similar to fedpkg in Fedora). Everybody who is able to commit to SCL dist-git is able to upload sources into the look-aside cache.

Example of work for internal RHSCL maintainers

# getting sources from centos into working directory of RH dist-git
$ rhpkg clone python
$ cd python
$ git remote add centos https://git.centos.com/scls/python.git
$ git fetch centos

# switch into centos branch
$ git checkout -b scl-python33-el7 --track centos/scl-python33-el7

# now it is time to do local commits
$ vim *.spec
$ rhpkg commit

# you may also upload files into centos look-aside cache
$ centpkg new-sources new.tar.gz

# pushing changes back to centos (do not forget about 'centos' here)
$ git push centos

Or alternative scenario using centpkg:

# do the changes directly in the centos in one directory
$ centpkg clone python
$ cd python
$ vim *.spec
$ centpkg commit
$ centpkg push

# then merge the changes in a directory cloned using `rhpkg`
$ rhpkg clone python
$ cd python
$ rhpkg remote fetch-sources centos # if RHBZ#973550 is implemented
$ git cherry-pick 7705c460
$ rhpkg push

FAQ

• Q: Where the packages will go (which repository)?
• A: answered bellow
• Q: How user will see the difference between "stable" packages and "development" packages?
• A: Proposal: there will be two repositories. One repository with stable downstream rebuild of RHSCL, another with upstream SCLs that include fixes above RHSCL and have quality compared to Fedora.