|
|
| Line 1: |
Line 1: |
| [http://www.brianlane.com Personal homepage] | | [http://www.brianlane.com Personal homepage] |
|
| |
|
| == Packages ==
| | [https://src.fedoraproject.org/user/bcl Fedora packages I am maintainer of, or involved with] |
| | |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/pylint pylint] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/python-logilab-common python-logilab-common] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/python-logilab-astng python-logilab-astng] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/python-astroid python-astroid] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/gnupg gnupg] fedora package (note, v1.4.10 not gnupg2)
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/livecd-tools livecd-tools] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/parted parted] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/bip bip] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/mx mx] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/lorax lorax] co-maintainer with [[User:wwoods | Will Woods]]
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/python-ipdb python-ipdb] fedora package
| |
| * [https://admin.fedoraproject.org/pkgdb/acls/name/maildrop maildrop] co-maintainer
| |
|
| |
|
| == Misc == | | == Misc == |
| Line 23: |
Line 10: |
| == Anaconda Development == | | == Anaconda Development == |
|
| |
|
| Development System
| | * grab the relevant boot.iso for the release you are working on |
| * Fedora 18 | | https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Everything/x86_64/os/images/boot.iso |
| * polipo proxy to cache packages
| | (I use Everything because it has, well, everything) |
| * lighttpd to serve up updates.img
| |
| * tftpd to serve up pxe boot images
| |
| * mock or lorax for building packages and iso's
| |
| | |
| == mock setup ==
| |
| | |
| * <code> yum install mock </code>
| |
| * Edit the <code>/etc/mock/site-defaults.cfg</code> file to bind your working directories inside the mock root. This keeps mock from wiping those directories on --init
| |
| <pre>
| |
| config_opts['plugin_conf']['bind_mount_enable'] = True
| |
| config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/home/user/projs/anaconda', '/root/anaconda' ))
| |
| config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/local/repo', '/repo' ))
| |
| config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/local/pungi', '/pungi' ))
| |
| </pre>
| |
| * Edit the <code>/etc/mock/</code> file for the distribution or copy it to a new name and edit
| |
| ** Add <code>proxy=http://proxy.home:3128</code> to the main section.
| |
| ** comment out mirrorlist entries
| |
| ** Open up the mirror list url(s) in a browser and pick a mirror, use the same one for base and updates
| |
| ** Add <code>baseurl=<mirror url></code> in each section instead of mirrorlist
| |
| ** Do this for each section with a mirrorlist
| |
| * Setup the initial mock environment, replace the fedora-13-i386 with the mock file edited above
| |
| ** <code>mock -r fedora-13-i386 --init</code>
| |
| ** <code>mock -r fedora-13-i386 --no-clean --install pungi</code>
| |
|
| |
|
| Now you have a mock chroot environment setup that will use the proxy cache for packages.
| | * Boot the iso in a VM and check the anaconda version in the boot.iso |
| | This version can be used to set where in history the makeupdates script starts grabbing changes from. |
| | Use virt-viewer->Send key->Ctrl+Alt+F1 to switch to the tmux console, the version will be at the top |
|
| |
|
| == Anaconda Debugging ==
| | * Branch anaconda from the release you are working on |
| | Make some changes (eg. start out by adding a log.info("FOO WAS HERE") to ./anaconda right before the |
| | anaconda._intf.setup(ksdata) line. |
|
| |
|
| === pdb ===
| | * Run ./scripts/makeupdates to generate an updates.img |
| | If you're curious about what it put into it, run with -k and it will keep the ./updates/ directory with all |
| | the files it put into it. |
| | If the version on the boot.iso is older than the most recent anaconda tag (shown by git log) you will |
| | need to tell the script that, by running it like: |
|
| |
|
| Insert this into Anaconda at the point where you want to start debugging:
| | ./scripts/makeupdates -t anaconda-29.18-1 |
| * <code>import pdb; pdb.set_trace()</code>
| |
|
| |
|
| When the breakpoint is hit you will be able to use pdb on tty1.
| | (the boot.iso I have is .18 and the current branch is tagged as .19 so you need to include |
| | all changes since .18 in the updates.img) |
|
| |
|
| === winpdb ===
| | * Setup a webserver to serve up the updates.img |
| | This can be as simple as: mkdir /tmp/updates; cp updates.img /tmp/updates; cd /tmp/updates; python3 -m http.server |
| | You'll need to know the IP of your workstation, and may need to open up port 8000 so the VM can access |
| | the webserver. You can do this with firewalld by by running: |
|
| |
|
| http://winpdb.org is a remote python debugger with support for threads.
| | firewall-cmd --add-port=8000/tcp && firewall-cmd --permanent --add-port=8000/tcp |
|
| |
|
| Insert this into Anaconda at the point where you want to start debugging:
| | * Then reboot the VM with the boot.iso, and interrupt the boot screen (I hit the up arrow to select the |
| * <code>import rpdb2; rpdb2.start_embedded_debugger('snakes', fAllowRemote=True)</code>
| | first entry), hit tab to edit it, delete 'quiet' to enable boot message output, and add: |
|
| |
|
| Create an update image with the rpdb2.py file:
| | updates=http://IP-TO-DEVBOX:8000/updates.img |
| * <code>./scripts/makeupdates -k</code>
| |
| * <code>cp /usr/lib/python2.7/site-packages/rpdb2.py ./updates/run/install/updates/</code>
| |
| * <code>./scripts/makeupdates -k</code>
| |
|
| |
|
| Boot your iso with the updates=http://url/of/your/updates.img and change to tty2. Run <code>ip addr</code> to find the IP of the system/virt.
| | And hit enter to boot with the updates.img |
|
| |
|
| When Anaconda hits the breakpoint run winpdb on your local system and use File->Attach to connect to the system and start debugging. The password (snakes) is set by the start_embedded_debugger call.
| | After it boots switch to the tmux console and check /tmp/anaconda.log for your new log message (FOO WAS HERE) |
|
| |
|
| Anaconda's storage imports appear to interfere with rpdb2's import wrapper so don't expect to be able to step through the whole program, but it is very useful for examining the state of the system and looking at the local variables for all the running threads.
| | If it doesn't boot it is likely a problem reading the updates.img, which can be a networking issue (VM not able |
| | to connect to the host) or the port (8000 in this case) isn't open. |
|
| |
|
| == Building boot images == | | == Building boot images == |
|
| |
|
| === with lorax ===
| | See the [http://weldr.io/lorax/lorax.html Lorax documentation] |
| From the top level of a checked out lorax git repo I run:
| |
| <pre>
| |
| PATH=./src/sbin/:$PATH PYTHONPATH=./src/ ./src/sbin/lorax -p Fedora -v 17 -r 17 \
| |
| -s http://download.fedora.redhat.com/pub/fedora/linux/development/rawhide/x86_64/os/ \
| |
| --proxy http://proxy.brianlane.com:3128 --config ./lorax.conf ./tmp/
| |
| </pre>
| |
| | |
| The -s points to repositories to use, add your own with higher nvr to override packages in the primary repo.
| |
| | |
| lorax.conf points to the template directory: | |
| <pre>
| |
| [lorax]
| |
| sharedir=/home/bcl/Red_Hat/projs/lorax/share
| |
| </pre>
| |
| | |
| results will be in ./tmp/
| |
| | |
| === with pungi (old) ===
| |
| | |
| Use the compose and pungi.ks files below, place them into /root/ in the mock to be built. Replace REPO_URL_HERE with the same repo you selected for the mock's yum.conf file.
| |
| | |
| * pungi.ks
| |
| <pre>
| |
| # un-comment the applicable repo
| |
| | |
| repo --name=fedora --baseurl=REPO_URL_HERE --proxy=http://proxy.home:3128
| |
| | |
| # Very small install footprint
| |
| %packages
| |
| @base
| |
| kernel
| |
| syslinux
| |
| nomtools
| |
| anaconda
| |
| %end
| |
| </pre>
| |
| | |
| * compose
| |
| <pre>
| |
| #!/bin/bash
| |
| echo "*** Running pungi --force --nosource --nodebuginfo --nosplitmedia -G -C -B -c /root/pungi.ks --ver=13"
| |
| | |
| pungi --cachedir=/pungi/cache --force --nosource --nodebuginfo --nosplitmedia -G -C -B -c /root/pungi.ks --ver=13
| |
| echo "*** Done."
| |
| </pre>
| |
| * <code> chmod a+x compose </code>
| |
| * <code> exit </code>
| |
| * <code> mock -r fedora-13-i386 --chroot "/root/compose" </code>
| |
| | |
| Change the --ver to match whichever release you are building for
| |
| | |
| == Building updates ==
| |
| | |
| Anaconda includes the ability to update itself by passing <code>updates=http://path/to/update.img</code> to the kernel at boot time. This allows you to use the same boot media and test changes to stage2 of the installer.
| |
| | |
| * This assumes a working mock chroot (ie. I built boot images with this one first)
| |
| * You need a git repo of Anaconda, I branch for the build so I don't clutter up master with build by-products.
| |
| ** <code> git clone git://git.fedorahosted.org/anaconda.git </code>
| |
| ** <code> cd anaconda; git checkout -b build-updates </code>
| |
| * bind the anaconda directory to the mock being used by editing <code>/etc/mock/site-defaults.cfg</code> as described above.
| |
| ** '''NOTE: DO NOT''' use <code>mount -o bind</code> to mount directories inside the mock. They will be obliterated the next time --init is run.
| |
| * Install the development tools group
| |
| ** <code> mock -r fedora-13-i386 --install @development-tools</code>
| |
| ** <code> mock -r fedora-13-i386 --install yum-utils</code>
| |
| * enter the chroot
| |
| ** <code> mock -v -r fedora-13-i386 --shell </code>
| |
| ** grab a copy of the current anaconda*src.rpm from someplace and install the dependencies with:
| |
| *** <code> yum-builddep anaconda*src.rpm </code>
| |
| ** now build anaconda
| |
| *** <code> ./autogen.sh </code>
| |
| *** <code> ./configure </code>
| |
| *** <code> make updates </code>
| |
| ** <code> exit </code>
| |
| * copy the updates.img over to the updates directory and optionally rename it
| |
| ** <code> cp /var/lib/mock/fedora-13-i386/root/root/anaconda/updates.img /home/tftpboot/images/updates/ </code>
| |
| | |
| You can skip installing development tools and anaconda deps for subsequent iterations.
| |
| | |
| Now add <code> updates=http://proxy.home/updates/updates.img </code> to the kernel parameters when booting the install media.
| |
| | |
| NOTE: By running yum inside the chroot you may mess up the rpmdb version, this depends on how close your host system is to the target system. So YMMV
| |
| | |
| == Build with a test anaconda.rpm ==
| |
| | |
| '''NOTE: This doesn't actually seem to work when the released anaconda version matches that in the branch. buildinstall isn't letting the local repo override the one in the repo'''
| |
| | |
| Normally when a punji build is done it pulls anaconda from the repo/proxy cache. Instead you want it to use your new build (ie. when doing stage1 development which cannot be updated by updates= being passed to the kernel)
| |
| | |
| * make sure mock is mounting your anaconda directory as described above. Again, do not use <code>mount -o bind</code> to do it.
| |
| * <code> mock -v -r fedora-13-i386-proxy --shell </code>
| |
| * <code> cd /root/anaconda </code>
| |
| * Removed the cached files. Otherwise it pull pull from there and not update to the latest
| |
| ** <code> rm -rf /13 </code>
| |
| ** <code> rm -rf /pungi/cache/local </code>
| |
| * <code> ./autogen.sh </code>
| |
| * <code> ./configure </code>
| |
| * <code> make scratch </code>
| |
| * <code> rpmbuild -ts --nodeps anaconda...tar.gz </code>
| |
| * <code> yum-builddep /builddir/build/SRPMS/anaconda...src.rpm </code>
| |
| * <code> rpmbuild --rebuild /builddir/build/SRPMS/anaconda...src.rpm </code>
| |
| * <code> mkdir /repo </code>
| |
| * <code> cp /builddir/build/RPMS/anaconda...rpm /repo </code>
| |
| * <code> createrepo /repo </code>
| |
| * edit <code> /root/pungi.ks </code> and add
| |
| ** <code> repo --name=local --baseurl=file:///repo/ </code>
| |
| * <code> exit </code>
| |
| * <code> mock -v -r fedora-13-i386 --chroot "/root/compose" </code>
| |
| * <code> rsync -avc /var/lib/mock/fedora-13-i386/root/13/i386/os/images/ /home/tftpboot/images/fedora/13/i386/ </code>
| |
| | |
| Boot with the pxe image and see what happens.
| |
| | |
| NOTE: The removal of the local repo cache is needed because yum gets confused by its presence.
| |
| | |
| == Update boot.iso with new anaconda rpm ==
| |
| | |
| After you have a working boot.iso you can easily update it with the files from the new anaconda rpm you built above. Use the [[http://git.fedorahosted.org/git?p=anaconda.git;a=blob_plain;f=scripts/upd-bootiso;hb=HEAD upd_bootiso]] script to do this:
| |
| | |
| <code>upd_bootiso boot.iso anaconda-13.35-1.fc13.i686.rpm</code>
| |
| | |
| This will extract the files from initrd.img and install.img on the boot.iso, update the files and then re-build the boot.iso
| |
| | |
| This is considerably faster than using mock + pungi to compose a whole new iso from scratch.
| |
| | |
| Note, this script currently only works with x86 due to the fact that I borrowed part of it from the <code>mk-images.x86</code> script and haven't built on other architectures yet.
| |
| | |
| == Releasing anaconda ==
| |
| | |
| This requires permission to upload new anaconda files and a [https://fedora.transifex.net/projects/p/anaconda/ Transifex account] with access to the Anaconda project. The transifex client needs to be setup on your system. This is described in the [http://git.fedorahosted.org/git/?p=anaconda.git;a=blob;f=docs/transifex.txt;hb=HEAD doc/transifex.txt] file in the anaconda source tree:
| |
| | |
| * git clean -d -x -f
| |
| * <code>./autogen.sh && ./configure</code>
| |
| ** If that fails for deps: <code>yum-builddep anaconda.src.rpm</code>
| |
| ** Alternatively grab the srpm using <code> fedpkg srpm </code> and run yum-builddep on that.
| |
| * <code> make bumpver </code>
| |
| * Make sure anaconda.spec.in looks ok
| |
| * <code> git commit -m "New version." </code>
| |
| * <code> make release </code>
| |
| * <code> git push </code>
| |
| * <code> git push --tags </code>
| |
| | |
| In the fedora package then do:
| |
| * <code> fedpkg switch-branch fXX </code>
| |
| * Copy the anaconda.spec and anaconda-*tar.gz over to the package directory
| |
| * Make sure anaconda.spec looks ok
| |
| * <code> fedpkg new-sources anaconda-XX.XX-X.tar.gz </code>
| |
| * <code> fedpkg clog </code>
| |
| * Edit the clog to fix the first line, make it consistent with the "- [text]" changelog entries.
| |
| * <code> git add anaconda.spec </code>
| |
| * <code> git commit -F clog </code>
| |
| * <code> git push </code>
| |
| * <code> fedpkg build </code>
| |
| * <code> fedpkg update </code>
| |
|
| |
|
| == Creating a disk image == | | == Creating a disk image == |
|
| |
|
| livemedia-creator makes it easy to create a disk image for use with virtual systems. It can also be PXE booted on a diskless system.
| | See the [http://weldr.io/lorax/livemedia-creator.html livemedia-creator documentation] |
| | |
| * <code>sudo livemedia-creator --make-fsimage --iso=/path/to/boot.iso --title Fedora --project Fedora --releasever 20 --vnc vnc --ks=/usr/share/doc/lorax/fedora-livemedia.ks |& tee lmc-build-out.log</code>
| |
| | |
| == PXE boot disk image ==
| |
| | |
| If you have enough memory (size of the initrd + normal system requirements) you can PXE boot a fsimage. The fsimage is compressed and appended to the end of the existing initrd for the release you want to boot. Create the new initrd using the [http://bcl.fedorapeople.org/scripts/img2pxe.sh img2pxe.sh] script.
| |
| | |
| * <code>img2pxe.sh /path/to/fedora/20/initrd.img /path/to/fsimage.img /path/to/initrd-disk.img</code>
| |
| | |
| Edit your PXE setup and create a menu entry that has an APPEND line like this:
| |
| | |
| * <code>APPEND initrd=/path/to/initrd-disk.img root=live:/squashfs.img rd.live.image</code>
| |
|
| |
|
| == Moving a project to github == | | == Moving a project to github == |
Personal homepage
Fedora packages I am maintainer of, or involved with
Misc
Anaconda Development
- grab the relevant boot.iso for the release you are working on
https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Everything/x86_64/os/images/boot.iso
(I use Everything because it has, well, everything)
- Boot the iso in a VM and check the anaconda version in the boot.iso
This version can be used to set where in history the makeupdates script starts grabbing changes from.
Use virt-viewer->Send key->Ctrl+Alt+F1 to switch to the tmux console, the version will be at the top
- Branch anaconda from the release you are working on
Make some changes (eg. start out by adding a log.info("FOO WAS HERE") to ./anaconda right before the
anaconda._intf.setup(ksdata) line.
- Run ./scripts/makeupdates to generate an updates.img
If you're curious about what it put into it, run with -k and it will keep the ./updates/ directory with all
the files it put into it.
If the version on the boot.iso is older than the most recent anaconda tag (shown by git log) you will
need to tell the script that, by running it like:
./scripts/makeupdates -t anaconda-29.18-1
(the boot.iso I have is .18 and the current branch is tagged as .19 so you need to include
all changes since .18 in the updates.img)
- Setup a webserver to serve up the updates.img
This can be as simple as: mkdir /tmp/updates; cp updates.img /tmp/updates; cd /tmp/updates; python3 -m http.server
You'll need to know the IP of your workstation, and may need to open up port 8000 so the VM can access
the webserver. You can do this with firewalld by by running:
firewall-cmd --add-port=8000/tcp && firewall-cmd --permanent --add-port=8000/tcp
- Then reboot the VM with the boot.iso, and interrupt the boot screen (I hit the up arrow to select the
first entry), hit tab to edit it, delete 'quiet' to enable boot message output, and add:
updates=http://IP-TO-DEVBOX:8000/updates.img
And hit enter to boot with the updates.img
After it boots switch to the tmux console and check /tmp/anaconda.log for your new log message (FOO WAS HERE)
If it doesn't boot it is likely a problem reading the updates.img, which can be a networking issue (VM not able
to connect to the host) or the port (8000 in this case) isn't open.
Building boot images
See the Lorax documentation
Creating a disk image
See the livemedia-creator documentation
Moving a project to github
- Make a new repo there (in rhinstaller or wherever it is going)
- Edit .git/config and make a copy of the origin section, replace the url with the new one.
- rename the old origin to something descriptive like 'fedorahosted' and also change the fetch line so the origin is replaced by this new name.
- on master run git push -u origin master to push it to the new location on github
- git push --tags
- git checkout rhel7-branch && git push -u origin rhel7-branch
- Repeat for any other active branches.
- git fetch fedorahosted
- git checkout -b fh-master fedorahosted/master
- git rm -r * && git rm .gitignore
- create and edit a dead.repo file
- git add dead.repo
- git commit -m "Moved XX to github"
- git push --dry-run
- If the destination looks correct, git push and if not, git push fedorahosted HEAD:<branchname>
