CallumLerwick/MockHacking

NOTE TO SELF: This is very outdated, update it.

This is a page detailing my personal mock setup, intended for local package testing and Extras package reviews.

The current default configs are old and buggy. Here's what mine look like:

CallumLerwick/MockHacking/I386Config CallumLerwick/MockHacking/X8664Config

These contain setup for a local repository. This is useful for reviewing a chain of dependencies, and for customizing the buildroot. Creating a local repo is easy. Create the directory tree:

mkdir /storage/localrpms
cd /storage/localrpms
mkdir i386 x86_64 noarch SRPMS

Then install the createrepo package. I use a script to maintain it:

#!/bin/sh
cd /storage/localrpms/||exit 1

mv -v *.src.rpm SRPMS/
createrepo -c .cache SRPMS/

mv -v *.noarch.rpm noarch/
createrepo -c .cache -g buildroots.xml noarch/

mv -v *.i[3456] 86.rpm i386/
createrepo -c .cache i386/

mv -v *.x86_64.rpm x86_64/
createrepo -c .cache x86_64/

This allows you to slap a bunch of rpms in the /storage/localrpms directory, (like when a build finishes) and the script will move them into the right places and update the metadata.

An example yum repo file for the local repo:

[local] 
name=local - $basearch
baseurl=file:///storage/localrpms/$basearch
enabled=1
gpgcheck=0

[local-noarch] 
name=Local noarch
baseurl=file:///storage/localrpms/noarch/
enabled=1
gpgcheck=0

/storage is NFS shared across all my local systems, I can test local packages on any of my systems with just a "yum install foo", and easily keep them in sync.

In the noarch repo, I put a local buildroots.xml. This lets me pull in ccache, which significantly speeds up repeated builds. (This requires patching mock to be functional, which we do next) It looks like this:

<?xml version="1.0"?>
<!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">

<comps>
<group>
<id>local</id>
<uservisible>true</uservisible>
<name>local</name>
<packagelist>
<packagereq type="mandatory">ccache</packagereq>
</packagelist>
</group>
</comps>

I have patched mock to be able to mount --bind directories into the chroot, allowing their contents to be preserved across builds. This allows you to preserve the yum cache, so you don't have to run squid or keep a local mirror. And it also preserves the ccache. Here's the patch:

diff -ubr mock-0.4/mock.py mock-0.4.bind/mock.py
--- mock-0.4/mock.py    2006-03-21 15:44:24.000000000 -0600
+++ mock-0.4.bind/mock.py       2006-05-16 00:30:38.000000000 -0500
@@ -177,6 +177,10 @@
self._umount('proc')
if os.path.exists('%s/%s' % (self.rootdir, 'dev/pts')):
self._umount('dev/pts')
+        if os.path.exists('%s/%s' % (self.rootdir, 'var/cache/yum')):
+            self._umount('var/cache/yum')
+        if os.path.exists('%s/%s' % (self.rootdir, 'builddir/.ccache')):
+            self._umount('builddir/.ccache')

if os.path.exists(self.basedir):
cmd = '%s -rfv %s' % (self.config['rm'] , self.basedir)
@@ -370,6 +374,7 @@
def _mount(self):
"""mount proc and devpts into chroot"""
mf = os.path.join(self.statedir, 'mounted-locations')
+        if os.path.exists(mf): return
track = open(mf, 'w+')

@@ -397,12 +402,37 @@
track.write('dev/pts\n')
(retval, output) = self.do(command)
track.flush()
-        track.close()

if retval != 0:
if output.find('already mounted') == -1: # probably won't work in other LOCALES
raise RootError, "could not mount /dev/pts error was: %s" % output

+        # yum cache
+        yumcachedir = os.path.join(self.rootdir, 'var/cache/yum')
+        self._ensure_dir(yumcachedir)
+        self.debug("mounting yumcache in %s" % yumcachedir)
+        command = '%s --bind /var/lib/mock/yum %s' % (self.config['mount'] , yumcachedir)
+        track.write('var/cache/yum\n')
+        (retval, output) = self.do(command)
+        track.flush()
+
+        if retval != 0:
+            if output.find('already mounted') == -1: # probably won't work in other LOCALES
+                raise RootError, "could not mount /var/cache/yum error was: %s" % output
+
+        # ccache
+        ccachedir = os.path.join(self.rootdir, 'builddir/.ccache')
+        self._ensure_dir(ccachedir)
+        self.debug("mounting yumcache in %s" % ccachedir)
+        command = '%s --bind /var/lib/mock/ccache %s' % (self.config['mount'] , ccachedir)
+        track.write('builddir/.ccache\n')
+        (retval, output) = self.do(command)
+        track.flush()
+        track.close()
+
+        if retval != 0:
+            if output.find('already mounted') == -1: # probably won't work in other LOCALES
+                raise RootError, "could not mount /builddir/.ccache error was: %s" % output

def _umount(self, path):

@@ -586,7 +616,7 @@
def _make_our_user(self):
-        if not os.path.exists(self.rootdir + self.homedir):
+        if not os.path.exists(os.path.join(self.rootdir + self.homedir, '.bash_profile')):
if not os.path.exists(os.path.join(self.rootdir, 'usr/sbin/useradd')):
raise RootError, "Could not find useradd in chroot, maybe the install failed?"
cmd = '/usr/sbin/useradd -u %s -d %s %s' % (self.config['chrootuid'] ,
Only in mock-0.4/src: libselinux-mock.so
Only in mock-0.4/src: mock-helper
diff -ubr mock-0.4/src/mock-helper.c mock-0.4.bind/src/mock-helper.c
--- mock-0.4/src/mock-helper.c  2005-07-16 11:21:33.000000000 -0500
+++ mock-0.4.bind/src/mock-helper.c     2006-05-14 14:00:16.000000000 -0500
@@ -231,6 +231,11 @@
else if (strncmp (rootsdir, argv[5] , strlen (rootsdir)) != 0)
error ("devpts: mount not allowed on %s", argv[5] );
}
+  else if (strncmp ("--bind", argv[2] , 6) == 0)
+  {
+    if (strncmp (rootsdir, argv[4] , strlen (rootsdir)) != 0)
+      error ("bind: not allowed on %s", argv[4] );
+  }
else
error ("unallowed mount type");

Only in mock-0.4/src: selinux-mock.o

You will probably have to "mkdir /var/lib/mock/yum /var/lib/mock/ccache" for it to work, and make sure the permissions are right.

TODO: Notice the repo names in my configs are namespaced by arch. I have been advised to change it to put the yum cache in /var/lib/mock/yum-cache/[configname] / , which is probably a better way to prevent different archs from stepping on each other's cache.

TODO: The code should be cleaned up to be more general and configurable.

DONE: Merge in autocache as well.

TODO: Investigate using distcc. (distcc seems to be stalled in review at the moment, though)

I launch mock with a wrapper script:

#!/bin/sh
sudo su build -c "CCACHE_DIR=/var/lib/mock/ccache/ ccache -z"
sudo su build -c "mock $*"
CCACHE_DIR=/var/lib/mock/ccache/ ccache -s
sudo rm -v /var/lib/mock/fedora-5-*-core/result/*.src.rpm \
&&sudo mv -v /var/lib/mock/fedora-5-*-core/result/*.rpm /storage/localrpms/ \
&&/storage/localrpms/update

This script takes care of sudoing to the "build" user to protect against malicious packages. (My own account is NOT even in the mock group) It clears the ccache stats first, runs mock, and displays the ccache stats afterwards. It then moves the resulting packages (except for source) into the local repo.