Mock Setup Using Local Mirror
Author: Parag Nemade Zdeng
Revision: 0.4
Initial Draft: Friday Jun 30, 2006
Last Revised: Saturday Aug 21, 2006
Last Modified: Wednesday Nov 12, 2014
Note: This page describes my personal mock setup using local mirror setup for FC6.
It is still under construction and content may change. Do not rely on the information on this page.
Setting Local Mirrors For Fedora Development version
Prior required steps are to have at least 10GB free space on your hard drive for setting Fedora development repository.
To setup local mirror, create following files
1)copy /etc/yum.repos.d/fedora-development.repo to /etc/yum.repos.d/mirror-fedora-development.repo
2)copy /etc/yum.repos.d/fedora-extras.repo to /etc/yum.repos.d/mirror-fedora-extras.repo
Say if you have /repo directory having minimum free space 10GB then download fedora-mirror.py from http://www.fedoraproject.org/wiki/Projects/fedora-mirror to /mirror_repo.
Change /srv/mirror to /mirror_repo in fedora-mirror.py file. For development version we need only development, extras and groups repository. We don't need any updates repository as its not exist.
Modified version of fedora-mirror.py
Final Modified version of fedora-mirror.py for development version is
#!/usr/bin/python
# License: GPL
# Author: Rudolf Kastl
# default target: /mirror_repo
# how?: use config file and special mirror .repo files
# todo:
## code:
# - ConfigParser
# - logging
# - legacy and updates-testing awareness
# - fix hierachy into the fedora mirror dir scheme
# - use separate configs for each release
# - progressbars
# - eta (complete, per repo, per file(?!))
## support files/packaging/integration:
# - creation of specially crafted .repo files
# - cron integration
# - 0 conf apache serving as split package
# - mock config files as split package
# - yum localmirror config files as split package
# - discuss access rights on /mirror_repo
################
# special cases:
# - the fedora core repos differ in logic -> core repos have updates, updates-testing, legacy -> to be investigated
# - development is another special case not having updates updates-testing and legacy
# - other repos (3rd party) never have updates and updates-testing
# - SRPMS are another special case -> same for each arch of the same release
###########################
import os
flavourlist=["core", "extras", "updates"]
versionlist=["development"]
archlist=["i386"]
subflavourlist=["binary"]
### block for parsing commandline switches
from optparse import OptionParser
parser = OptionParser()
# var can be accessed with options.filename
parser.add_option("-d", "--dest", dest="destination", help="Sync Destination", metavar="dir", default="/mirror_repo")
parser.add_option("-n", "--newonly", action="store_true", dest="newonly", help="Sync Destination", default=False)
parser.add_option("-t", "--testonly" , action="store_true", dest="testonly", help="No Sync Transaction", default=False)
parser.add_option("-q", "--quiet", action="store_false", dest="verbose", default=True, help="don't print status msgs to stdout")
(options, args) = parser.parse_args()
### end bloc
targetlocation = options.destination
def syncMirror (version, flavour, arch, subflavour, targetlocation):
print "syncing: Fedora " + version, flavour, arch, subflavour, "to", targetlocation
# handling the special cases
if subflavour == "binary":
subflavourid = ""
subflavourpath = ""
elif subflavour == "source":
subflavourid = "-"+subflavour
subflavourpath = "/SRPMS"
else:
subflavourid = "-"+subflavour
subflavourpath = "-"+subflavour
# devel is special
if version == "development":
repoid="mirror-"+flavour+"-"+version+subflavourid
else:
repoid="mirror-"+flavour+subflavourid
if subflavour == "source":
targetdir=targetlocation+"/fedora/"+version+"/"+flavour+subflavourpath
else:
targetdir=targetlocation+"/fedora/"+version+"/"+flavour+"/"+arch+subflavourpath# handling the newonly option of reposync
if options.newonly == True:
newonly = " -n"
else:
newonly = ""
# building the command for syncing a single repo
syncdistro="mirrorreleasever="+version+" mirrorbasearch="+arch+" /usr/bin/reposync"+newonly+" -r "+repoid+" -p "+targetdir
# for testing we just print instead of sync
if options.testonly == True:
print syncdistro
else:
# create potentially missing paths
try:
if os.path.isdir(targetdir) == False:
os.makedirs(targetdir)
except:
print "Error: Cannot create targetdirectory:", targetdir
os.system(syncdistro)
## main loop
for version in versionlist:
for flavour in flavourlist:
for arch in archlist:
for subflavour in subflavourlist:
# stuff that just doesent happen
if version == "development" and flavour == "updates":
continue
if version == "development" and flavour == "legacy":
continue
if version == "development" and flavour == "updates-testing":
continue
else:
syncMirror(version, flavour, arch, subflavour, targetlocation)
Creating Directories for Local Repository Path
Now start downloading all development repository to /mirror_repo by giving
python fedora-mirror.py
Once all core and extras development repository downloaded, Now move core and extras repository to parent directory /mirror_repo.
fedora-mirror.py will download core repository at
/mirror_repo/fedora/development/core/i386/mirror-core-development
fedora-mirror.py will download extras repository at
/mirror_repo/fedora/development/extras/i386/mirror-extras-development
Create following directories
/mirror_repo/fedora/development
/mirror_repo/fedora/development/i386/
/mirror_repo/fedora/development/i386/core
/mirror_repo/fedora/development/i386/extras
/mirror_repo/fedora/development/i386/groups
Setting up core repository directory
Then create clean i386 repository by doing following
mv /mirror_repo/fedora/development/core/i386/mirror-core-development/* /mirror_repo/fedora/development/i386/core
cd /mirror_repo/fedora/development/i386/core
createrepo .
Setting up extras repository directory
mv /mirror_repo/fedora/development/extras/i386/mirror-extras-development /mirror_repo/fedora/development/i386/extras
cd /mirror_repo/fedora/development/i386/extras
createrepo .
Setting up groups repository directory
cd /mirror_repo/fedora/development/i386/groups
create minimal.xml and add its contents as
<?xml version="1.0"?> <!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd"> <comps> <group> <id>build</id> <uservisible>true</uservisible> <name>Minimal Install</name> <packagelist> <packagereq type="mandatory">bash</packagereq> <packagereq type="mandatory">bzip2</packagereq> <packagereq type="mandatory">coreutils</packagereq> <packagereq type="mandatory">cpio</packagereq> <packagereq type="mandatory">diffutils</packagereq> <packagereq type="mandatory">fedora-release</packagereq> <packagereq type="mandatory">gcc</packagereq> <packagereq type="mandatory">gcc-c++</packagereq> <packagereq type="mandatory">gzip</packagereq> <packagereq type="mandatory">make</packagereq> <packagereq type="mandatory">patch</packagereq> <packagereq type="mandatory">perl</packagereq> <packagereq type="mandatory">rpm-build</packagereq> <packagereq type="mandatory">redhat-rpm-config</packagereq> <packagereq type="mandatory">sed</packagereq> <packagereq type="mandatory">tar</packagereq> <packagereq type="mandatory">unzip</packagereq> <packagereq type="mandatory">buildsys-macros</packagereq> </packagelist> </group> </comps>
download buildsys-build and buildsys-macors rpms under /mirror_repo/fedora/development/i386/groups
wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-build-0.5-1.noarch.rpm
wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-build-0.5-3.fc6.noarch.rpm
wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-macros-6-2.fc6.noarch.rpm
wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-macros-6-2.fc6.src.rpm
Final contents of directory groups will look as
ls /mirror_repo/fedora/development/i386/groups
minimal.xml buildsys-macros-6-2.fc6.noarch.rpm buildsys-macros-6-2.fc6.src.rpm buildsys-build-0.5-1.noarch.rpm buildsys-build-0.5-3.fc6.noarch.rpm
now do createrepo -g minimal.xml . under /mirror_repo/fedora/development/i386/groups
Thus you will find all repository set under /mirror_repo/fedora/development/i386.
Mock Setup for Local Mirror
I have setup mock environment for development(FC6) version. My mock configuration for i386 is /etc/mock/fedora-devel-i386-core.cfg
import os config_opts['root'] = 'fedora-development-i386-core' config_opts['target_arch'] = 'i386' config_opts['yum.conf'] = """ [main] cachedir=/var/cache/yum debuglevel=1 reposdir=/dev/null logfile=/var/log/yum.log retries=20 obsoletes=1 gpgcheck=0 assumeyes=1 [core] name=core baseurl=file:///mirror_repo/fedora/development/i386/core [updates-released] name=updates baseurl=file:///mirror_repo/fedora/development/i386/core [groups] name=groups baseurl=file:///mirror_repo/fedora/development/i386/groups [extras] name=extras baseurl=file:///mirror_repo/fedora/development/i386/extras [local] name=local baseurl=http://extras64.linux.duke.edu/plague-results/fedora-5-extras enabled=0 """
Building packages in Mock Environment
To build packages in mock you need to go through http://fedoraproject.org/wiki/PackageMaintainers/MockTricks
Troubleshooting for fedora-mirror.py
If you face any problems using fedora-mirror.py script then you can use /usr/bin/reposync command to download binary RPMS for extras repository.
Core repository can be setup by copying FC6 installation DVD/CDROMs to /mirror_repo/fedora/development/i386/core. Detailed Procedure to do this can be found at http://www.city-fan.org/tips/YumRepoFromImages
reposync command without any options will download extras repository. Just move all those RPMS to /mirror_repo/fedora/development/i386/extras and use createrepo . command
Thus still you can download and create local repository without fedora-mirror.py script.
Mock Setup Using Local Mirror For EPEL (taking RHEL6.x as a example)
Set Up Local Mirror For RHEL/Centos Development
RHEL yum online update is not free, i.e. we could not install any packges using yum in RHEL. So we need to setup other repos or using local mirror for repo.
Remove the orginal yum installation
# rpm -qa|grep yum|xargs rpm -e --nodeps warning: /etc/yum.conf saved as /etc/yum.conf.rpmsave
Downloading Centos yum packages
python-iniparse：http://mirror.centos.org/centos-6/6.4/os/i386/Packages/python-iniparse-0.3.1-2.1.el6.noarch.rpm
yum-metadata-parser：http://mirror.centos.org/centos-6/6.4/os/i386/Packages/yum-metadata-parser-1.1.2-16.el6.i686.rpm
yum：http://mirror.centos.org/centos-6/6.4/os/i386/Packages/yum-3.2.29-40.el6.centos.noarch.rpm
yum-plugin-fastestmirror：http://mirror.centos.org/centos-6/6.4/os/i386/Packages/yum-plugin-fastestmirror-1.1.30-14.el6.noarch.rpm
RPM-GPG-KEY：http://mirror.centos.org/centos-6/6.4/os/i386/RPM-GPG-KEY-CentOS-6
# wget http://mirror.centos.org/centos-6/6.4/os/i386/Packages/python-iniparse-0.3.1-2.1.el6.noarch.rpm # wget http://mirror.centos.org/centos-6/6.4/os/i386/Packages/yum-metadata-parser-1.1.2-16.el6.i686.rpm # wget http://mirror.centos.org/centos-6/6.4/os/i386/Packages/yum-3.2.29-40.el6.centos.noarch.rpm # wget http://mirror.centos.org/centos-6/6.4/os/i386/Packages/yum-plugin-fastestmirror-1.1.30-14.el6.noarch.rpm # wget http://mirror.centos.org/centos-6/6.4/os/i386/RPM-GPG-KEY-CentOS-6
Install the downloaded yum packages
Note: Import RPM-GPG-KEY-CentOS-6 before yum install
yum and yum-plugin-fastermirror need to be install together
# rpm -ivh python-iniparse-0.3.1-2.1.el6.noarch.rpm # rpm --import RPM-GPG-KEY-CentOS-6 # rpm -ivh yum-metadata-parser-1.1.2-16.el6.i686.rpm # rpm -ivh yum-3.2.29-40.el6.centos.noarch.rpm yum-plugin-fastestmirror-1.1.30-14.el6.noarch.rpm
Now, we can usring yum for online installation.
Setup Local Mirror Using ISO Image
Mount on the ISO Image
# mount -t iso9660 -o loop /mnt/RHEL-6.6-20140926.0-Workstation-i386-dvd1.iso /mnt/RHEL-6.6-Localrepo
Create repo file
# vi /etc/yum.repos.d/local.repo ------------------------------------- [rhel6] name=rhel6-local baseurl=file:///mnt/RHEL-6.6-Localrepo gpgcheck=1 gpgkey=file:///mnt/RHEL-6.6-Localrepo/RPM-GPG-KEY-redhat-release enabled=1
You can use yum clean all and yum makecache to update the repos. Now the local repo is ready for use.
Mock Setup for Local Mirror for EPEL
My mock configuration file is "epel-6-i386".Make configuration as following:
# vi /etc/mock/epel-6-i386.cfg
-----------------------------------
config_opts['root'] = 'epel-6-i386'
config_opts['target_arch'] = 'i686'
config_opts['legal_host_arches'] = ('i386', 'i586', 'i686', 'x86_64')
config_opts['chroot_setup_cmd'] = 'groupinstall buildsys-build'
config_opts['dist'] = 'el6' # only useful for --resultdir variable subst
...
...
...
[local]
name=local
baseurl=file:///mnt/RHEL-6.6-Localrepo
cost=2000
enabled=1
...
...
Now mock can using local mirror for packages installations.