Architectures/ARM/AArch64

Have a question? Join the Fedora ARM team on IRC in #fedora-arm on Freenode

Introduction

ARM chips have been wildly successful in embedded applications, cell phones and tablet devices -- but none of those has gotten to be more than a 32-bit system. ARM has recently decided to bring their low power consumption expertise into the server market; there are efforts underway from several vendors for ARMv7 Cortex A9 or Cortex A15 based server systems. ARM have decided to make it really interesting in servers, though, by introducing ARMv8 (the AArch64 architecture), a new 64-bit architecture derived from the history and experience of the previous 32-bit systems.

You can find out more at ARMv8 Architecture.

This page documents the initiative to bring Fedora to this processor family, starting with the bootstrapping of Fedora 17.

Bootstrap Process

Essentially, we are following the same process as for Fedora 15 on ARM: ARM HardFP Bootstrap.

The purpose of the effort is to enable Fedora on ARMv8, using the standard ABI as defined by ARM. The good news is that unlike ARMv7 and earlier, there is only one ABI.

Goals

Our goals are to complete the bootstrap "stages" (1,2,3,4,5,6) at which we anticipate certain levels of functionality from the root filesystem, beginning with very basic cross compilation, proceeding to native builds, and ultimately to a working mock and Koji configuration able to build production packages.

CURRENT STAGE: We have completed stage1, and are progressing through stage2.

Bootstrap Tools

Again capitalizing on the HardFP port work previously done, we re-used and modified as necessary several scripts written earlier by DJ Delorie. These scripts are stored in a git repository and can be retrieved as follows:

$ git clone http://fedorapeople.org/groups/armv8/bootstrap.git

Patches are of course welcome.

Stage 1 - make

Status: mostly complete

Stage 1 was an initial bootstrap using cross compilation from source (not from SRPMs) on x86_64 hosts, for a minimal set of aarch64 packages. The goal was to create a minimal chroot environment which was sufficient for building the more complex packages of stage2. This implies providing native versions of GCC, binutils, glibc, plus some basic tools such as gzip, patch and awk, along with a few others. Stage1 results include these tools and a BusyBox environment that provides most of the normal and expected Linux commands. It does not include packages beyond the very minimal set required to get to a native building stage (still without RPM).

If you have access to cross-compilers for ARMv8 in your PATH, and those cross-compilers are named aarch64-redhat-linux-gnu-*, you should be able to recreate all of stage1 with the following:

$ git clone http://fedorapeople.org/groups/armv8/bootstrap.git
$ cd bootstrap
$ mkdir f17-local
$ scp username@fedorapeople.org:/project/armv8/SRPMs/f17/changed/* f17-local
$ ln -s f17-local SRPMlocal
$ mkdir f17-sources
$ scp username@fedorapeople.org:/project/armv8/SRPMs/f17/unchanged/* f17-sources
$ ln -s f17-sources SRPMs
$ ./stage1

If you do not have cross-compilers with names of the form given above, you will need to edit the stage1 script to point at the right places.

Further information:

• Stage1 Notes
• Task list:
  • [ahs3] Package and make available the Linaro aarch64 cross-toolchain [INPROGRESS]
  • Kernel needs to be brought up to date -- either upstream 3.7 with patches, Linaro AArch64, or something similar [TODO]
  • Binutils has had several aarch64 patches added in the last few weeks [TODO]
  • GCC patches should be re-examined for possible updates [TODO]
  • Glibc patches probably need to be updated to latest upstream [TODO]
  • Investigate and resolve additional rpath info needed for cloog. Why? [TODO]
  • Investigate and resolve additional rpath info needed for libselinux. Why? [TODO]
  • Investigate and resolve additional rpath info needed for sed. Why? [TODO]
  • [ahs3] Make sure any changes made go back into upstream, where appropriate [INPROGRESS]

Stage 2 - rpmbuild

Status: in progress

Stage 2 has the goal of getting a working native build of RPM (rpmbuild) running within the rootfs. This includes enough packages built from source (not from SRPMs, but on the target) to get a working "rpmbuild" binary sufficient to rebuild RPM and other packages. The output from this is the ability to build native RPM packages.

Further information:

• Stage2 Notes
• How To Use the ARM Foundation Model -- instructions for using what is currently the only publicly available platform simulator for ARMv8 with the rootfs built in stage1 (which is how one builds stage2 results)
• Task list (these all need to be built in dependency order):
  • [ahs3] publish patchify scripts [INPROGRESS]
  • Gdb still FTBFS -- newer upstream patches available? [TODO]
  • [ahs3] Upstream patches? Perl, nspr, to date [INPROGRESS]
  • [ahs3] Gettext FTBFS [INPROGRESS]
  • Build chkconfig [TODO]
  • Build sqlite [TODO]
  • Build pth [TODO]
  • Build openssl [TODO]
  • Build popt [TODO]
  • Build libffi [TODO]
  • Build pkgconfig [TODO]
  • Build glib2 [TODO]
  • Build nss-util [TODO]
  • Build nss-softokn [TODO]
  • Build nss [TODO]
  • Build elfutils [TODO]
  • Build rsync [TODO]
  • Build fakechroot [TODO]
  • Build file [TODO]
  • Build ustr [TODO]
  • Build libusb1 [TODO]
  • Build libusb [TODO]
  • Build tzdata [TODO]
  • Build redhat-rpm-config [TODO]
  • Build rpm [TODO]
  • Build cracklib [TODO]
  • Build libsepol [TODO]
  • Build attr [TODO]
  • Build acl [TODO]
  • Build libsemanage [TODO]
  • Build pam [TODO]
  • Build shadow-utils [TODO]
  • Build strace [TODO]
  • Build filesystem [TODO]
  • Build iso-codes [TODO]
  • Build audit [TODO]
  • Build gnupg [TODO]
  • Final re-build of rpm [TODO]

Stage 3 - mock

Status: TODO

Stage 3 is the effort to get a minimal set of useful buildroot RPMs sufficient to rebuild the buildroot and have a working yum, mock configuration. RPMs will be manually built with rpmbuild and uploaded to a shared central rootfs repository.

At stage3, it may be necessary to intentionally disable certain unnecessary features of RPMs, and GNU autotools (and so on) will automatically exclude certain features dependent on non-present functionality. Consequently, these RPMs will not be final and must be rebuilt in the following stages.

Stage 4 - distribution bootstrap

Status: TODO

At Stage 4 we will run a complete mock rebuild of the distribution, identifying and solving dependency and platform build issues. With those out of the way, we will switch over to koji while waiting for the koji and yum changes needed to correctly support the new architecture type.

The result of stage4 is threefold:

1. The set of packages needed for running Koji
2. A set of SRPMs which actually can build some form of F15, or sufficiently large part of it.
3. A set of binary RPMs that can be used initially by koji while doing the first koji rebuild of the distribution

Stage 5

Status: TODO

Complete rebuild of the distribution using koji

Stage 6

Status: TODO

At this point we should have koji-shadow running automatically shadowing the main koji.

Getting Involved

There are various ways you can get involved. If in doubt, visit the Fedora ARM IRC (#fedora-arm on irc.freenode.net) and ask one of us for advice.

Who's involved?

• Al Stone
• Jon Masters
• Chris Tyler
• DJ Delorie
• David Marlin
• Dennis Gilmore
• Brendan Conoboy
• Peter Robinson
• Your Name Could Go Here!

Task Lists

In the sections above are task lists for the various stages. These are the ones we currently know about. If you have further suggestions, let us know on IRC or the fedora-arm mailing list where we can discuss them and figure out how they fit into the overall scheme of things. As part of that discussion, we'll decide if and how it gets added to the lists. As you'll notice, there is a marker in front of each task (well, sometimes) and a marker at the end of each task (always). The marker at the end should be one of TODO (meaning the task has not been started by anyone), INPROGRESS (implying someone is actively working on it), or DONE (um, well, the task is done :). If the task is INPROGRESS or DONE, there should be a marker at the front indicating who is doing the work (e.g., ahs3 is Al Stone's nick on IRC).

If you're looking for something to do, find a task that is marked TODO, and edit this page to show that you're working on it. If someone has put their nick on the task, please ask them first and share nicely like grown-ups. For example, change the line:

   * grizzle the fingo [TODO]

to this:

   * [myNick] grizzle the fingo [INPROGRESS]

And when you've completed the task, change it to:

   * [myNick] grizzle the fingo [DONE]

Git Based Rootfs Workflow

For stages 1 through 3, we are coordinating work using the git repository at http://fedorapeople.org/groups/armv8/rootfs.git.

IMPORTANT: Please do work using only the ARM Foundation model. While cross-compilers are possible, we are trying to stay as close as possible to doing a target-based (i.e., real hardware) bring up at this time. We use cross-compilation in stage1, but we are trying to be very close to the "Fedora way" of operating, which is build on target. For the time being, the ARM Foundation model is as good as it gets. As soon as hardware becomes available, we will make this more stringent and insist all stage2 and later work being done on real hardware.

IMPORTANT: Build flags, and similar options are defined in the stage1 and stage2 scripts. In stage1/stage2, please do not enable features that are not required for minimal bootstrap, and especially DO NOT enable Thumb2, or NEON, and so forth in any contributed builds. In stage3 onward, do not change build flags and such as those that are defined globally in the RPM packages (rpm, redhat-rpm-config, fedora-release...).

• http://fedorapeople.org/groups/armv8/rootfs.git -- just pull the master branch

Our workflow is centered around the above git repository -- a copy of the current state of the rootfs. You will need to clone this git repository and then build a disk image from it; instructions can be found in How To Use the ARM Foundation Model. If you can do so and make your git clone publicly visible, you can then send pull requests to the fedora-arm list for review and incorporation. If you cannot make your clone publicly visible, please send a git diff of the SRPM changes made to the fedora-arm list; after review and incorporation, we will build from the source changes and incorporate the binaries back into the rootfs also (sending the binaries to the list would just be painful).

Assisting with Stage1

The results of the first three stages of the bootstrap are being kept in a rapidly growing and quite large git repository that can retrieved via:

$ git clone http://fedorapeople.org/groups/armv8/rootfs.git

This was seeded with stage1 results and the current stage2 results that were built using the scripts found here:

$ git clone http://fedorapeople.org/groups/armv8/bootstrap.git

All of the source that was used can be found at http://fedorapeople.org/groups/armv8/SRPMs/f17. The directory unchanged contains the SRPMs that are unchanged from the source one can get from the Fedora 17 sources. The directory changed contains all the Fedora 17 (or in some cases, 18) SRPMs that had to change to build.

If you have access to cross-compilers for ARMv8 in your PATH, and those cross-compilers are named aarch64-redhat-linux-gnu-*, you should be able to recreate all of stage1 with the following:

$ git clone http://fedorapeople.org/groups/armv8/bootstrap.git
$ cd bootstrap
$ mkdir f17-local
$ scp username@fedorapeople.org:/project/armv8/SRPMs/f17/changed/* f17-local
$ ln -s f17-local SRPMlocal
$ mkdir f17-sources
$ scp username@fedorapeople.org:/project/armv8/SRPMs/f17/unchanged/* f17-sources
$ ln -s f17-sources SRPMs
$ ./stage1

If you do not have cross-compilers with names of the form given above, you will need to edit the stage1 script to point at the right places.

If you wish to contribute changes to the stage1 SRPMs or the bootstrap scripts, please submit either a git diff for the package needing to be changed, or a git pull request from your clone of the repository, to the fedora-arm mailing list for review and ultimately incorporation into the proper git repository by the maintainer (ahs3).

NB: better access to RPMs for the x86_64 cross-toolchain for aarch64 is being worked on. In the meantime, tools can be found at Linaro.

Assisting with Stage2

Stage2 builds upon the rootfs originally created in stage1 and found here:

$ git clone http://fedorapeople.org/groups/armv8/rootfs.git

To use this rootfs as a native build environment, we have to use the platform simulator provided by ARM -- the ARM Foundation model. Instructions on using the model with the rootfs are found here: How To Use the ARM Foundation Model. Once you have booted the model with the rootfs, you'll be presented with a console running BusyBox. From there, running stage2 is very straightforward:

$ cd /stage2
$ ./stage2

The stage2 script uses makefile snippets defined /stage2/recipe.d and stage2/recipe.mk to order which packages get built in what order. You can modify that order by either (a) cheating and marking something as done by creating the proper file in /stage2/done (e.g., 'touch /stage2/done/gdb' lets stage2 continue past the build for gdb even though it currently fails), or (b) the more difficult route of modifying the makefile snippets directly.

If you wish to contribute changes to the stage2 SRPMs or the bootstrap scripts, please submit a git diff for the package needing to be changed, or a git pull request from your clone of the repository, to the fedora-arm mailing list for review and ultimately incorporation into the proper git repository by the maintainer (ahs3).

Subsequent Stages

Stage3 will look a lot like a larger stage2, and hopefully have more hands involved. And while it is okay at this point to disable sub-packages like "docs", or to turn off functionality like selinux, or audit requirements, do not disable features that will be required by other packages directly in order to get a minimally working mock buildroot established. If in doubt, ask on #fedora-arm for advice.

Starting in stage4, we will be able to drop the use of the rootfs from git and work from real images. Stages 4-6 will start to look more like a normal Fedora release, hopefully.

Target flags

No special flags are required at this time.

Hardware floating point

Hardware floating point in ARMv8 processors is considered standard and not optional. Hence, no special options are required.