Thoughts on rpkg2

New rpkg2
=========

.. attention::
   This document is a summary of recent discussion of new rpkg2. This document
   does not aim to be a design document of new rpkg2. Content of each section
   has been keeping updated and also would be changed in the future.

Background
----------

``rpkg`` is a generic Python package to build package tools that builds package
with dist-git and Koji. It is used to develop a few of package tools for
various software communities, e.g.  ``fedpkg`` for Fedora, ``centpkg`` for
CentOS, ``rdopkg`` for RDO, and ``rhpkg`` for Red Hat internal package build.
Some of them are built on top of rpkg by extending core class to add specific
functionalities, and others, e.g. rdopkg, just call ``fedpkg`` to build
packages in Koji.

``rpkg`` provides a set of core features, aka. commands, that allows packagers
to manage package's sources files, test changes by building package locally, and
do either scratch build or normal build in Koji. Meanwhile, downstream package
tools built on top of ``rpkg`` could add their own commands and extend the
command line interface so that packagers are able to use the new features.

rpkg2 will be a rewritten version of rpkg totally with a proposed new
architecture design. Major goals are

* to reduce duplicate code when creating downstream package tools

* to make it more reusable for creating CLI for downstream package tool

* to make it easier to reuse rpkg by other applications to access package
  information

* to make it easier to extend, customize and add new features according to
  specific community's requirement.

For all discussion about this topic so far, please refer to issue `Propose new
mechanism to refactor cli codebase`_.

.. _Propose new mechanism to refactor cli codebase: https://pagure.io/rpkg/issue/49

Thoughts about new architecture
-------------------------------

In new architecture, original rpkg will be modularized and each of them is
dedicated for handling specific tasks. rpkg2 will be

* a framework of CLI to allow developers to create CLI application easily
  without copy and paste.

* a set of components (or they could be called libraries, modularize modules,
  whatever the name) that can be used to get various information from package
  and its git repository and interact with backend package infrastructure like
  dist-git and lookaside. They may include

  * **PackageRepo**: interface to get underlying git metadata, e.g. tags, branches

  * **Package**: interface to get package information (the metadata), e.g. module
    name, NVR, disttag, distval, distvar, epoch, giturl.

  * **Distgit**: interface to operate package repository, e.g. clone, commit, push

  * **Koji**: interface to interact with Koji to build packages, e.g. scratch-build,
    build, mock-config

  * **Lookaside**: interface to upload and download sources files

  * **LocalBuild**: interface to build package locally

  * **TestToolkit**: provide basic helper classes and methods to help developer
    to write test cases for downstream package tool easier.

  all these could be reuse by any other applications.

* a framework to allow developers to write new commands for their package tool
  easily.  A new command can be written, distributed and installed outside rpkg
  as long as it follows some rules defined by rpkg2 so that it is able to be
  discovered and loaded dynamically in the runtime.

  A little more details here

  * rpkg2 provides a set of core commands dedicated to package build and
    operations with Lookaside, which are good for sharing with downstream
    package tools.

  * downstream package tool developers can write their own commands, and those
    commands can be totally new, a customized based on core commands, or
    disabled.

  Currently, this would benefit fedpkg and rhpkg at least.

Development
-----------

Some development information of rpkg2 would be

**Name**: rpkg2

**Repository**: pagure.io. It would be nice to have a place like github's
organization, for example https://pagure.io/packagetool/rpkg2. There would be
fedpkg2 https://pagure.io/packagetool/fedpkg2

**RPM**: python2-rpkg2

**PyPI**: rpkg2

Each source code file should declare its license

rpkg2 should use ``koji`` as much as possible to avoid writing duplicate code
of ``koji``.

rpkg2 should be compatible with Python 3 by default.

It would be nice to have a complete Jenkins pipeline from patch to RPM and
PyPI, so that everyone who is interested in rpkg2 can try it easily.

Use py.test instead of nose, since nose has been in maintenance for the past
several years.

Tests should run under both Python 2 and 3. Python 2.6 is the minimum version
to support.  Python 3.6 is maximum supported version, since it is in Fedora
rawhide as of writing this.

Do not use `GitPython`_, and use git command directly instead. Major reasons

* GitPython does not support some core git feature, AFAIK which is worktree so
  far.  IIRC, someone requested to support worktree for packaging, and it is
  really a useful feature for packager.

* In rpkg, GitPython is usually used to get config items from repository, which
  can be migrated to a subprocess call against git command simply.

* Other popular Python package for git is `pygit2`_, whose documentation tells
  "It is likely to work with Python 2.6 and 3.1, but these versions are not
  officially supported.", but supporting Python 2.6 is still required for rpkg2.

However, GitPython can still be used in tests.

.. _GitPython: http://gitpython.readthedocs.io/en/stable/
.. _pygit2: http://www.pygit2.org/

What you can do with rpkg2
--------------------------

Some use cases that could be done with rpkg2.

Get package version from package repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

An application could import rpkg2 and call a method to get the version. Before
calling, the application has to ensure repository's path has been passed
properly.

.. code:: python

    from rpkg2.packagerepo import PackageRepo
    version = PackageRepo('path/to/repo').ver

Write a new command document-build
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

New command ``document-build`` would be used for building product documentation
in Koji, and it should be added to downstream package tool fedpkg. But, it is
not a good idea to make it available along with fedpkg's installation, since
this command is only useful for a small range of users and installing many
dependent packages would be a noise for other packagers. So, as a developer, I
can do following

* start a separate repository as a new project

* write some code to implement command ``document-build``

* hacking, testing, hacking testing ...

* distribute this new project in either Python package or RPM

* install the package

Now, command ``document-build`` should be available and can be used with fedpkg::

    fedpkg document-build -h

Known Issues
------------

Here lists some known issues that rpkg2 should fix them.

* `#164`_ Reuse bash completion

.. _#164: https://pagure.io/rpkg/issue/164