From Fedora Project Wiki
Resources
IRC Log
| -!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - An introduction to mock, the chroot package builder with your teacher ( nirik ) - See https://fedoraproject.org/wiki/Classroom for todays class schedule and more info. | 14:45 | |
| @nirik | welcome back everyone. | 14:45 |
|---|---|---|
| @nirik | Our next talk is an Introduction to mock. | 14:45 |
| DiscordianUK | Hi | 14:45 |
| @nirik | with your teacher... me. ;) | 14:45 |
| @nirik | So, I am gonna go over what mock is, why it's needed and how to use it, then some tricks and tips. | 14:46 |
| sdodson | Sounds good. | 14:46 |
| @nirik | feel free to ask questions at anytime. | 14:46 |
| @nirik | ok, so some intro. | 14:47 |
| @nirik | mock is a package that makes a chroot and builds other packages in it. | 14:47 |
| @nirik | it's available in any of the fedora repos. You can install it with a simple 'yum install mock' | 14:47 |
| @nirik | You can also build packages other releases with it. For example, if you have F10 and want to build a rawhide package you can, or if you have f10 and want to build a f9 package. | 14:48 |
| @nirik | mock downloads and installs all the needed packages in it's chroot to build your package. | 14:48 |
| @nirik | It has a bunch of caching ability, so you only usually suffer a large download the first time you build something. I will talk about the cache later. | 14:49 |
| @nirik | So, why do we want to go to all this trouble? why not just build the package locally and be done with it? | 14:49 |
| @nirik | The reason is rerproducability. Some packages look for things at compile time. We have to know that when we build something it's reproducable and we can build it the same way again. | 14:50 |
| @nirik | so, we have a known set of base packages that everything gets and then after that, packages must specify exactly what they need to build. | 14:50 |
| @nirik | The Fedora build system uses mock to build each package that maintainers build. | 14:50 |
| @nirik | any questions on what mock is or why we have it? | 14:51 |
| @nirik | ok, moving on... | 14:51 |
| Ritmo2k | Yeah, wo its the std base set of packages that make it advantagous over a non root build? | 14:51 |
| @nirik | ah, excellect question. | 14:51 |
| @nirik | for fedora, look at http://fedoraproject.org/wiki/Packaging/Guidelines and the buildrequires section... | 14:52 |
| zcat | what was used before mock? custom rpmbuild scripts? | 14:53 |
| @nirik | basically it's a group called 'buildsys-build' | 14:53 |
| @nirik | so if you look at that group in yum you can see exactly whats in it. | 14:53 |
| @nirik | zcat: yeah, I think it was a simpler version in the buildsys.... | 14:54 |
| DiscordianUK | so what if yu're | 14:54 |
| DiscordianUK | so what if what you're building exceeds the buildsys-build specs? | 14:54 |
| @nirik | despite mock being made for building packages you can also use it for testing or making populated chroots. I will talk about that a bit later as well. | 14:54 |
| @nirik | DiscordianUK: then you need to add to your package spec file a "BuildRequires: whatever you need" | 14:55 |
| DiscordianUK | ok | 14:55 |
| @nirik | it will then install those things as well and their dependencies. | 14:55 |
| DiscordianUK | cool | 14:55 |
| @nirik | ok, so moving on to some simple usage. If anyone would like to follow along, you can 'yum install mock' first. | 14:55 |
| @nirik | the first thing you want to do is add your user (or the user you want to use mock) to the 'mock' group so it can run the command. | 14:56 |
| @nirik | usermod -a -G mock myusername | 14:56 |
| @nirik | for example | 14:56 |
| @nirik | you may need to logout and back on to get the new group or run 'newgrp mock' after that. | 14:56 |
| @nirik | After that you can run mock. The easiest command that most people will use is 'rebuild' | 14:57 |
| @nirik | so, you do 'mock rebuild my.src.rpm' | 14:57 |
| @nirik | when you issue that command, mock creates a chroot of your current fedora version, installs the buildsys-build group into it, then installs the src.rpm's BuildRequirers packages and all their dependencies into it, | 14:58 |
| Ritmo2k | how do specify where the chroot takes place in the event the deps are huge and need lotsa space? | 14:58 |
| @nirik | it then calls the rpmbuild in the chroot and rebuilds the package. You should ideally end up with your binary and src.rpm packages at the end. | 14:58 |
| @nirik | Ritmo2k: good question. By default mock uses /var/lib/mock/ and cache in /var/cache/mock. You can specify other dirs on the command line. | 14:59 |
| @nirik | or you can edit the mock config file in /etc/mock/ | 14:59 |
| @nirik | /etc/mock/site-defaults.cfg is the config you want to change there. | 15:00 |
| @nirik | depending on your package mock could take up a fair bit of space. | 15:00 |
| @nirik | mock has some other base command line arguments as well... | 15:01 |
| @nirik | init - init a chroot, but don't do anything with it yet. | 15:01 |
| @nirik | chroot <cmd> - chroot to the mock chroot you made and run command <cmd> | 15:01 |
| @nirik | installdeps / install <pkg> - will install a package or a package and all it's deps into the chroot. | 15:02 |
| Ritmo2k | Can you elaborate? | 15:02 |
| @nirik | on which item? :) | 15:02 |
| Ritmo2k | The first, | 15:03 |
| @nirik | using those commands you can use mock to make a chroot that has whatever packages you like in it. | 15:03 |
| AndyCap | Hmm, my mock on F10 didn't have an /etc/mock/default.cfg | 15:03 |
| @nirik | on init? Basically it does the create the chroot step and installs buildsys-build into it. It doesn't build anything after that or do anything until you run other commands on that chroot | 15:04 |
| Ritmo2k | ok | 15:04 |
| @nirik | AndyCap: should be site-default.cfg | 15:04 |
| @nirik | AndyCap: but I am looking at rawhide, so it might be a difference with the f10 version. ;( | 15:04 |
| @nirik | so, when mock is running on say a rebuild of a src.rpm, it makes some log files with all the things it's doing. | 15:05 |
| AndyCap | nirik: what I did was symlink a distro-arch.cfg to default.cfg, at least that seemed to be what the manpage meant | 15:05 |
| @nirik | yeah, I would have to check a f10 machine. Not sure whats going on there. ;( | 15:05 |
| @nirik | any further questions on the base commands? AndyCap: I can get a f10 machine here after the talk and see whats going on for you... | 15:06 |
| AndyCap | I'll do fine. don't worry | 15:07 |
| @nirik | ok, so lets talk a bit about log files. ;) mock makes logs of it's actions, which are often quite helpfull. | 15:07 |
| @nirik | Since the fedora buildsys uses mock, we can take a look at an example there. | 15:07 |
| @nirik | lets take a look at: http://kojipkgs.fedoraproject.org/packages/rkhunter/1.3.4/4.fc11/data/logs/noarch/ | 15:07 |
| @nirik | these are the 3 log files mock makes on a build run. First state.log shows high level commands it's running. | 15:08 |
| @nirik | then root.log shows the packages it's installing in the chroot... both the buildsys-build and the deps. | 15:08 |
| @nirik | then build.log is the output of rpmbuild on the src.rpm in the chroot. | 15:09 |
| * nirik will pause for a sec and let everyone look over logs. Any questions on those? | 15:09 | |
| Ritmo2k | Wow, so do those build servers grab deps from a cached mirror :) That would amount to alot of dl'ing for every package! | 15:10 |
| @nirik | ok, lets move on to talking about mock's caching ability. | 15:10 |
| Ritmo2k | cached 'local' mirror | 15:11 |
| @nirik | Ritmo2k: they do. They have a caching proxy over a local mirror. ;) | 15:11 |
| @nirik | but mock also does some nice caching. ;) | 15:11 |
| @nirik | so mock can cache at 3 points (and does by default). | 15:11 |
| @nirik | You can see the buildsys-build download there was 113MB. It would be anoying to download that everytime you rebuilt your package. | 15:12 |
| @nirik | So, mock keeps a cache of the chroot it last used for your package. It can just unpack that again and apply any updates since it was last used, and use it for the build. | 15:12 |
| sdodson | That's per package? | 15:13 |
| AndyCap | mock uses a "clean" chroot for each build? | 15:13 |
| @nirik | sdodson: I think it's per os-arch. So, if you built a package with f9/i386 it would re-use that cache. | 15:13 |
| @nirik | you can of course tell it not to use the cache. | 15:14 |
| @nirik | AndyCap: unless it's using a cached one from a previous build. | 15:14 |
| @nirik | mock also uses yum to install things, so it can use your yum cache if you have that enabled to pull packages from. | 15:14 |
| Ritmo2k | So the chroot is a minimal installation of your target OS, then it adds what ever deps it needs? | 15:15 |
| @nirik | finally mock can use ccache for compiled binaries | 15:15 |
| @nirik | Ritmo2k: yes. | 15:15 |
| AndyCap | nirik: I guess what I'm asking is it unpacking a pristine cache like rpm does with sources or is it using the same directory if I were to build more than one package without asking mock to clean up? | 15:15 |
| @nirik | AndyCap: it does unpack a new one. What it does is the first time you compile a given os/arch it will make a root cache file... basically a tar.gz of the root | 15:16 |
| @nirik | then it you compile another thing using that same os/arch, it will unpackage that 'clean' root into the new build. | 15:16 |
| AndyCap | good, thank you. | 15:17 |
| @nirik | so there is always unpacking time/disk io involved. | 15:17 |
| @nirik | but if you are testing rebuilds of the same package over and over, the caching should really help you not use too much network BW on downloads. | 15:18 |
| @nirik | Here you can see an example of a mockbuild I did the other day... | 15:18 |
| @nirik | http://scrye.com/pastebin/196 | 15:18 |
| @nirik | this shows the typical output from a rebuild... | 15:19 |
| @nirik | you can see there that it used a existing cache for the root | 15:19 |
| @nirik | it also tells you how long the build took. | 15:20 |
| @nirik | any further questions on cache or the output? | 15:20 |
| @nirik | ok, moving along to some talk of the config files... | 15:21 |
| @nirik | mock has it's config under /etc/mock as I mentioned earier. There are config files for each os/arch... | 15:22 |
| @nirik | if you need to modify what repository you are using, or add a local repo for some packages not in fedora, thats where you do that. | 15:22 |
| @nirik | or if you need to get more debugging from yum or mock when running it to track down a problem. | 15:22 |
| @nirik | in those files you find the yum.conf that mock uses to install with. | 15:23 |
| @nirik | also, this is a good time to mention smock... it's a wrapper around mock that rwmjones made to allow you to use local repos easily in your builds. | 15:24 |
| @nirik | http://hg.et.redhat.com/cgi-bin/hg-misc.cgi/fedora-mingw--devel/file/tip/smock | 15:24 |
| @nirik | any questions on the config files? | 15:25 |
| @nirik | AndyCap: yeah, it looks like you should ln default.cfg to whatever os/arch config you want to use by default. | 15:26 |
| @nirik | ok, moving along then... | 15:26 |
| AndyCap | nirik: ah. read it again and I see you can choose release-arch with -r | 15:27 |
| @nirik | AndyCap: yes, you can choose which specific config you want to use with -r | 15:27 |
| @nirik | note that you can also use centos repos and build against centos4/centos5 with those configs. | 15:27 |
| @nirik | ok, so you can also use mock to generate chroots for testing packages or just whatever reason you might have to use a fedora chroot. | 15:28 |
| @nirik | you can use the init and installdeps and chroot for this. | 15:29 |
| @nirik | along with the 'shell' command to run a shell in the chroot. | 15:30 |
| @nirik | one thing worth noting, if you are running multiple mock commands you will want to use the --uniqueext option, so it makes different chroots for each instance. | 15:31 |
| @nirik | if you do that it should work fine to run multiple mock instances at a time. | 15:31 |
| * nirik checks his outline to see what else he wanted to cover... | 15:32 | |
| @nirik | any other general questions. | 15:33 |
| @nirik | Another thing to note is that if you have a x86_64 machine, you can use mock to build both i386 and x86_64 packages, simply choose the right config with -r when you run it. | 15:33 |
| @nirik | There are some more tips and suggestions at the mock tips page on the fedora wiki: https://fedoraproject.org/wiki/Extras/MockTricks | 15:34 |
| @nirik | sorry, thats an old link. You want: https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds | 15:35 |
| @nirik | 'man mock' has a good deal of info as well. | 15:35 |
| @nirik | ok, I think thats about all I was hoping to cover... | 15:36 |
| AndyCap | nirik: thank you. it was interesting | 15:37 |
| @nirik | no problem... mock is handy for building and testing things for sure. :) | 15:38 |
| AndyCap | and killing your laptop. :P | 15:38 |
| @nirik | depending on what you are building, yeah. ;) | 15:39 |
| baconfork | nirik: Thank you | 15:39 |
| sdodson | If you were working in an environment where the buildroot changes a lot, like a koji builder, is squid probably the best caching tool? | 15:39 |
| @nirik | sdodson: yeah, squid works fine. I think thats what the fedora buildsys uses to cache packages like that. | 15:40 |
| @nirik | all the ones in buildsys-build then are always hot in the cache. | 15:40 |
| sdodson | nirik: *nod* that doc you referenced has squid settings in it | 15:40 |
| sdodson | We have a koji hub that has horrible connectivity between it and the builders. | 15:41 |
| @nirik | yeah, really you can use whatever you like, anything that caches http and works with yum | 15:41 |
| @nirik | but squid is pretty quick and easy to setup to do this. | 15:41 |
| @nirik | ok, if there are no further questions, I guess we can close up this session. ;) | 15:42 |
| @nirik | next up is the wrap up session... which we can start anytime. | 15:42 |
| ndim | nirik: Would it be difficult to make mock not just build a chroot, but a KVM/qemu image for different arches? | 15:43 |
| @nirik | ndim: humm... it would need to make a image of the chroot and also install a boot loader, etc... but it should be possible. | 15:44 |
| -!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - Wrap up session - See https://fedoraproject.org/wiki/Classroom for todays class schedule and more info. | 15:44 | |
| @nirik | so, I guess we can take a break here, or folks can feel free to chime in now with any feedback. Suggestions for how to better run classes? ideas or comments? all feedback welcome. | 15:45 |
| @nirik | I'd like to thank everyone for coming. Please suggest new topics on the wiki, or better sign up to teach something you know! | 15:46 |
| * nirik notes its officially the start of the wrap up session time. Feel free to bring up any feedback... or catch herlo or I with any suggestions. | 16:01 | |
| -!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - Next Classes (tenative) 10:00 UTC 2009-04-04/2009-04-05 - See https://fedoraproject.org/wiki/Classroom more information. End user help /join #fedora | 16:12 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!