Architectures/AArch64/Bootstrap/Stage2Notes

AArch64 Bootstrap Stage2 Notes

Here's what changed from f17 to get aarch64 to bootstrap (using the rootfs built in stage1) in build order as determined by dependencies in the make file fragments. Stage1 details are here.

A copy of the current state of the stage2 rootfs can be retrieved from:

$ git clone http://fedorapeople.org/groups/armv8/rootfs.git

This rootfs can be used directly as an NFS root mount point if you have access to either the ARM FAST model or the ARM Foundation model (see http://www.arm.com/fvp); details for doing so are in Using ARMv8 Bootstrap Root FS via NFS root. With some preparation, you can also turn this set of files into a rootfs disk image that can be used with the ARM Foundation model (see ARMv8 using Fedora17 Bootstrap Image and ARM Foundation Model).

NB: just as with stage1 ( AArch64 Bootstrap Stage1 Notes), the sources used can be found here:

http://fedorapeople.org/groups/armv8/SRPMs

Look in the subdirectories f17/changed (which should have been copied into SRPMlocal in your copy of the bootstrap directory), and in the subdirectory f17/unchanged (which should be copied into SRPMs in your bootstrap directory).

To work with the stage2 rootfs, follow the instructions pointed to above and boot a model and either an NFS root or a disk image. Once you have booted into the rootfs, however, continuing the stage2 build is pretty straightforward:

# cd /stage2
# ./stage2

You will be running in a BusyBox environment at that point, and you do have most basic commands, and basic job control.

cpio

Source: cpio-2.11-6.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Patched use of gnulib to remove deprecated use of gets(). Added patches to cpio.spec.

ncurses

Source: ncurses-5.9-4.20120204.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to ncurses.spec.

m4

Source: m4-1.4.16-3.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Patched use of gnulib to remove deprecated use of gets(). Added patches to m4.spec.

flex

Source: flex-2.5.35-14.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to flex.spec.

bison

Source: bison-2.5-3.fc17.aarch.src.rpm

Patched aarch64 support to config.sub and config.guess. Patched use of gnulib to remove deprecated use of gets(). Added patches to bison.spec.

readline

Source: readline-6.2-4.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to readline.spec.

texinfo

Source: texinfo-4.13a-16.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to texinfo.spec.

gdb

Source: gdb-7.5.0.20120926-25.fc17.aarch64.src.rpm

Started with 7.5 version of the source (from f18), since the upstream patch uses ptrace and as such it did not apply to the 7.4 version; several functions ended up rewritten or replaced so that in some cases the upstream patch was referring to code that did not exist in 7.4.

Further, the upstream patch was incomplete. While it got most parts, it made the assumption that the subdirectories bfd/, opcodes/, and include/ had all been updated to reflect the latest binutils for aarch64; this was not the case for f17. So, merged each of those directories with the binutils files (in the same directories) from stage1 in order to add support for aarch64 everywhere it was needed.

Patched aarch64 support into config.sub and config.guess. Added all of these patches to gdb.spec.

[msalter] While this builds, the resulting gdb is badly broken. I tried using it to debug a hang in libnss3.so and found that gdb segfaults when first starting up. I tracked this down to an issue with how gdb uses a make rule to create an init.c which calls various initialization routines. This file should have dozens of calls but ends up with just a couple default calls. If I run the script in the makefile by hand, it seems to work, so I used that result to rebuild gdb. It got further but there was an internal error: "_initialize_gdb_osabi: gdb_osabi_names[] is inconsistent". This was caused by a "Newlib" entry added to the gdb_osabi_names[] array but no definition added for it in the header. I removed the entry from the table and rebuilt. Now gdb starts up, but when I run the inferior, I get a message that gdb can't get a process id for a given lwp due too a capability error. This looks like a problem in libthread_db.so.

[msalter] Updated to gdb-7.5.0.20120926-25.fc17.aarch64_1.src.rpm which uses a newer upstream patch. Even so, it appears that native debugging is not fully supported (the capability error mentioned above is a symptom of this). gdbserver support seems more complete and may be the only option for now. In any case, the 3.4 kernel which is downloaded with the Foundation model is too old to support the newer ptrace interface gdb/gdbserver expects. A newer kernel will be needed before gdb/gdbserver can be used.

libtool

Source: libtool-2.4.2-3.1.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to libtool.spec.

gdbm

Source: gdbm-1.10-2.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to gdbm.spec.

db4

Source: db4-4.8.30-10.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to db4.spec.

perl

Source: perl-5.14.2-215.fc17.aarch64.src.rpm

Patched the definition of Perl_repeatcpy() in util.c so that the last argument was consistently defined; it had been type IV (which defaulted to 64-bit on aarch64) in the definition, but type I32 (32-bit) in the declaration. Added patch to perl.spec. Enabled a second patch that correct the installation path from /usr/lib/perl5 to /usr/lib64/perl5.

curl

Source: curl-7.24.0-5.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to curl.spec.

nspr

Source: nspr-4.9.2-1.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to nspr.spec.

Patched _linux.cfg and _linux.h in pr/include/md to add in the aarch64 architecture; cloned the definitions used from ia64, another LP64 platform. Added patch to nspr.spec.

pcre

Source: pcre-8.21-6.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to pcre.spec.

Changed the recipe.d make snippet so that documentation is not installed. When running make install, the command lines generated (e.g., when installing the HTML files or man pages) are quite long and will sometimes get cut in odd places causing the make to fail, but only for the documentation; everything else has built properly.

gettext

Source: gettext-0.18.1.1-14.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to gettext.spec.

Repaired deprecated use of gets() in old versions of gnulib, added patch to spec file.

Everything compiles automatically just fine; the make install step fails. The cause appears to be overly complex shell code in some of the Makefiles created by ./configure (in particular in gettext-tools/src and gettext-tools/man). For reasons yet to be determined, these Makefiles will fail because portions of shell variables have been completely corrupted; in the gettext-tools/src case, a list of the executables to be installed in the install-binPROGRAMS step gets horribly mangled so it tries to install programs that of course do not exist ('omm', in one case, which I thought was kind of funny, because I was no longer calm since the name should have been 'excomm' :). Because the Makefiles are in error, and not the Makefile.in or Makefile.am, these would have to be patched after running the configure step; this can be fixed in the recipe.d snippet for gettext but for now I have created Makefiles called Makefile.hacked in the directories mentioned above and do the make install step by hand. So, if you want to rebuild gettext in stage2, let it do 'make', and let it get as far it can with 'make install'. When it does fail, go into the stage2/builds/gettext directory and where ever there is a Makefile.hacked file, copy that to Makefile. Then, cd to stage2/builds/gettext and run 'make install', which should now complete. When it does, 'touch stage2/done/gettext' to let the stage2 build continue to the next step.

UPDATE: glibc (glibc-2.16-17.fc17.aarch64_1.src.rpm) patch fixed the make install issue so manual intervention is no longer needed.

popt

Source: popt-1.13-10.fc17.aarch64.src.rpm

Patched aarch64 support to config.sub and config.guess. Added patch to popt.spec.

chkconfig

Source: chkconfig-1.3.59-1.fc17.src.rpm

This requires gettext and popt to be built first but did not have them as "#requires" in the recipe.d file; added them.

sqlite

Source: sqlite-3.7.11-3.fc17.aarch64.src.rpm

Patched aarch64 support into config.sub and config.guess. Added patch to sqlite.spec.

pth

Source: pth-2.0.7-15.fc17.aarch64.src.rpm

Patched aarch64 support into config.sub and config.guess. Added patch to pth.spec.

openssl

Source: openssl-1.0.0j-2.fc17.src.rpm

Add in lib64 as a possibility in the recipe.d/0020-openssl make snippet so correct library search paths are used.

popt

Source: popt-1.13-10.fc17.aarch64.src.rpm

Patched aarch64 support into config.sub and config.guess. Added patch to popt.spec.

libffi

Source: libffi-3.0.10-2.fc17.aarch64.src.rpm

Added in upstream patch created by ARM for libffi.

Patched aarch64 support into config.sub and config.guess. Added patch to libffi.spec.

pkgconfig

Source: pkgconfig-0.25-4.fc17.aarch64.src.rpm

Patched aarch64 support into config.sub and config.guess (two sets of them). Added patches to pkgconfig.spec.

glib2

Source: glib2-2.32.4-2.fc17.aarch64.src.rpm

This one was nastier than most. First, recipe.d/0023-glib2 contained a sed of Makefile.in that was a bit overzealous in what it changed, causing the Makefile produced to fail; that has been removed entirely in favor of a patch in the SRPM instead. Secondly, there were instances in glib/Makefile and gio/Makefile where the steps generated by automake for install steps failed when executed -- there appears to be a problem in using pipes via shell commands in makefiles that needs to be hunted down. Regardless, the net result would be that 'make install' would not work in some cases. For those cases, unlike in gettext above, we could create a patch to Makefile.in so that ./configure would work properly. And, finally, config.sub and config.guess both needed to support aarch64. Built a patch for all of the above, and added it to glib2.spec.

UPDATE: glibc (glibc-2.16-17.fc17.aarch64_1.src.rpm) patch fixed the make install issue so Makefile patches were removed in glib2-2.32.4-2.fc17.aarch64_1.src.rpm

nss-utils

Source: nss-util-3.13.6-1.fc17.src.rpm

No changes needed to the source tree. The recipe.d/0023-nss-utils file from bootstrap.git did need an additional export of USE_64=1 to force a 64-bit version to be built.

nss-softokn

Source: nss-softokn-3.13.6-1.fc17.aarch64.src.rpm

The recipe.d/0024-nss-softokn file from bootstrap.git did need an additional export of USE_64=1 to force a 64-bit version to be built.

There is an infinite loop somewhere in the shlibsign command being generated by this package. For the time being, I completely disabled the use of the tool as it is not really needed for the bootstrap. What the command does is generate checksum files that would be used to maintain FIPS140 certification; since we're not likely to need that any time soon during stage2, it was commented out and put in a patch to be un-applied at some point in the future.

nss

Source: nss-3.13.6-1.fc17.aarch64.src.rpm

The recipe.d/0025-nss file from bootstrap.git did need an additional export of USE_64=1 to force a 64-bit version to be built. Further, the installation of lib*.chk files was disabled in the recipe.d file; since these are not produced -- and are not needed for now -- it was commented out.

As with nss-softokn, there is an infinite loop somewhere in the shlibsign command being generated by this package. For the time being, I completely disabled the use of the tool as it is not really needed for the bootstrap. What the command does is generate checksum files that would be used to maintain FIPS140 certification; since we're not likely to need that any time soon during stage2, it was commented out and put in a patch to be un-applied at some point in the future.

elfutils

Source: elfutils-0.154-2.fc17.aarch64.src.rpm

Needed to update config.{guess,sub} to recognize aarch64; patched spec file to match.

NB: this is a basic port of elfutils; may need to go back and add aarch64 specific code in the future.

fakechroot

Source: fakechroot-2.9-26.fc17.src.rpm

No changes needed.

rsync

Source: rsync-3.0.9-2.fc17.aarch64.src.rpm

Updated config.{guess/sub} for aarch64.

ustr

Source: ustr-1.0.4-12.fc17.src.rpm

Patched recipe.d file to install in lib64. There were a couple of errors during install which needed fixing. First, install tried running /usr/sbin/ldconfig but ldconfig was in /sbin. /sbin should be a link to /usr/sbin in fc17, so I moved all files from /sbin to /usr/sbin and made /sbin a link. Second, ldconfig complained about /usr/lib64/libstdc++.so.6.0.17-gdb.py not being an ELF file. That file actually belongs in /usr/share/gdb/auto-load/usr/lib64/ so I moved it there.

NB: /sbin has now been moved to /usr/sbin and made a link to /usr/sbin. The libstdc++*py file has also been moved though it needs fixing in the gdb build, I think.

Update: /sbin problem was tracked to dev-template in stage1 script and fixed there. libstdc++.so.6.0.17-gdb.py was also a problem in stage1 script (gcc install step) which has also been fixed there.

libusb1

Source: libusb1-1.0.9-0.6.rc1.fc17.aarch64.src.rpm

Patched config.guess/config.sub for aarch64.

libusb

Source: libusb-0.1.3-10.fc17.src.rpm

No changes needed.

tzdata

Source: tzdata-2012f-1.fc17.src.rpm

Patched stage2/rpmbuild/BUILD/tzdata/Makeconfig to have reasonable values for objpfx and install_root variables. The sources in stage2/rpmbuild were created from "rpmbuild -bp" and this left objpfx with a path from the host running rpmbuild. It left install_root defined as @install_root@ because that text replacement isn't done until the install step in the spec file.

Changed the recipe.d file to sed in the proper paths into Makeconfig (as the spec file does) instead of patching the source. [ahs3]

redhat-rpm-config

Source: redhat-rpm-config-9.1.0-28.fc17.src.rpm

Patched recipe.d file so rpm scripts install to /usr/lib/rpm rather than /usr/lib64/rpm. Patched rpmrc to add aarch64 optargs entry.

rpm

Source: rpm-4.9.1.3-7.fc17.aarch64.src.rpm

Added aarch64 support patch.

cracklib

Source: cracklib-2.8.18-3.fc17.src.rpm

No changes needed.

libsepol

Source: libsepol-2.1.7-3.fc17.src.rpm

Patched recipe.d file to install in lib64. Also patched same file to override TARGET variable defined in local.conf and passed on make command line in stage2 script. The libsepol Makefile defines it's own TARGET variable so TARGET coming from local.conf had to be overridden.

attr

Source: attr-2.4.46-6.fc17.aarch64.src.rpm

First attempt failed during install:

make: Entering directory `/stage2/builds/attr/attr'
../include/install-sh -m 755 -d /usr/bin
/bin/sh ../libtool --mode=install ../include/install-sh -m 755 attr /usr/bin
../libtool: line 3646: install_prog+=../include/install-sh: not found
../libtool: line 3646: install_prog+= -m: not found
../libtool: line 3646: install_shared_prog+= -m: not found
../libtool: line 3646: install_prog+= 755: not found
../libtool: line 3646: install_shared_prog+= 755: not found
../libtool: line 3646: files+= attr: not found
libtool: install: you must specify an install program
libtool: install: Try `libtool --help --mode=install' for more information.

Investigation showed this to be very much like the problem with gettext builds described above. As seen above, /bin/sh is the shell used to run the libtool script. I could reproduce this by running the command by hand. If I substitute /bin/bash or run libtool directly, the install succeeds. So /bin/sh does indeed seem to be the problem here. The SHELL var used when invoking libtool is hardcoded to /bin/sh in stage2/rpmbuild/BUILD/attr-2.4.4/include/buildmacros. So a patch for this file was added so that SHELL is /bin/bash.

Nice catch on the problem being the shell; it had not occurred to me. Added SHELL=/bin/bash to recipe.d file instead of the source patch.

Update: updated glibc eliminates need for the SHELL override.

acl

Source: acl-2.2.51-6.fc17.aarch64.src.rpm

Need same SHELL var patch to buildmacros file as attr package. It would probably be better to just link /bin/sh to /bin/bash rather than using the busybox shell.

Added SHELL=/bin/bash to recipe.d file instead of the source patch.

Update: updated glibc eliminates need for the SHELL override.

libsemanage

Source: libsemanage-2.1.6-3.fc17.src.rpm

Patched recipe.d file to install in lib64. Also patched same file to override TARGET variable defined in local.conf and passed on make command line in stage2 script. The libsemanage Makefile defines it's own TARGET variable so TARGET coming from local.conf had to be overridden.

pam

Source: pam-1.1.5-7.fc17.aarch64.src.rpm

Patched config.{guess,sub} for aarch64. Patched recipe.d file to add --disable-nis to avoid build error due to missing glibc header file.

shadow-utils

Source: shadow-utils-4.1.5-4.fc17.aarch64.src.rpm

Patched config.{guess,sub} for aarch64.

strace

Source: strace-4.7-1.fc17.aarch64.src.rpm

Added aarch64 patch from Steve McIntyre which was posted to upstream strace list.

filesystem

Source: filesystem-3-2.fc17.aarch64.src.rpm

Patched spec file so that aarch64 links /lib to /usr/lib64 instead of /usr/lib for other arches.