Kernel/Rawhide

Rawhide

The following page will cover how we deal with the kernel in Rawhide. It's a mix of history, policy, and process. Essentially it can be summed up as:

It moves forward with upstream.

What and Why

The Rawhide kernel is essentially the latest git snapshot of Linus' upstream kernel.org tree. On a frequent (often daily) basis, a new snapshot is built. We do this for a variety of reasons, but the main reason is that we are building and testing (via the rawhide repo) the same kernel that upstream is currently developing on. This keeps us current and hopefully catches issues before they make it into a stable Fedora release. Of course, we are limited to the usage of rawhide itself, but we have nothing better at the moment.

Isn't it unstable?

Yes. There are definitely bugs, some of which even prevent machines from booting at times. However, in terms of process and quality, the upstream kernel is actually pretty remarkable. Major issues a fairly rare. Usually we can get a fixed build out within a week after engaging with upstream in these cases, often much faster.

"Policies"

There are a few things we do with rawhide kernels that aren't done at all with the other releases. These aren't set in stone rules, but over the years we've found they tend to help things so we continue to do them.

The first is that we leave debug options enabled. There's a whole page on this alone: https://fedoraproject.org/wiki/KernelDebugStrategy . However, as a somewhat nice gesture to our users, we turn them off with the first build of every major milestone. So 4.3-rc1, rc2, ... 4.3 final will all have debug options disabled. This brings back some performance. Other than that, debug options tend to stay on.

The second is that at times Rawhide will carry patches aside from the normal bugfixes that are not in other branches. Rawhide is the experimentation area, so we are slightly more lenient of high value additions that are still working details out upstream. These cases are fairly rare, but they do happen. As of Sept 2015, the kdbus patchset is the only "rawhide only" patchset we are carrying.

The last policy is around new driver enablement. This has varied over the years from "turn everything on" to "turn nothing on". Neither is really a great choice. Everything on leads to bloat, nothing on leads to no new hardware being supported. So we try and find a balance. Essentially, the breakdown looks like this:

• Networking drivers and options are almost always enabled and set to =m
• HID and input drivers are typically =m unless it seems like an obscure piece of hardware
• ARM options are handled by the ARM team. The kernel team literally guesses for a first cut. If an option relates to a board/mach the ARM team has enabled, it tends to be enabled in the appropriate section. If not, it tends to be disabled.
• GPIO/I2C/IIO drivers tend to be disabled by default. We somewhat rely on people asking for them to be enabled.
  • Of late, accel and ambient light drivers are of more interest.
• New filesystems tend to be disabled. New options to existing filesystems tend to get enabled, but reviewing with the relevant FS teams is good.
• USB is a mixed bag and decisions require reading what the option is actually for.

When in doubt, disabled or =m are the safe choices.

How

OK, so how does one roll out rawhide? It's pretty easy. Here is what you will need:

• The pkg-git checkout (duh)
• A clone of Linus' kernel tree. The master branch should track Linus' master branch and only that.
• Time
• Some kind of coping mechanism for all the options you will hit over and over during the merge window. Choice is up to you.

The steps below assume you have your pkg-git checkout in ~/kernel and your git tree in ~/src/linux. Adjust as appropriate.

git snapshot

1. Update both of your trees.
   1. cd ~/kernel/; git checkout master; git pull; pushd ~/src/linux; git checkout master; git remote update; git merge origin/master (or use git pull) ; pushd +1
2. If you're doing a new git snapshot, use the generate-git-snapshot script in pkg-git to create one.
   1. LINUX_GIT=~/src/linux ./scripts/generate-git-snapshot.sh
      • LINUX_GIT is the env variable that points to the tree to generate the patch from. This is why it is important to keep your exploded tree updated and tracking Linus' master.
      • That script will create a patch-<version>-git<N>.xz file in your local directory, and throw a changelog entry into kernel.spec.
   2. Upload the file: fedpkg upload patch-4.3-rc1-git1.xz
      • Edit the sources file to make sure it only lists the files needed
3. Run a prep on the tree
   1. fedpkg prep
      • You are highly likely to run into patches that don't apply any longer, etc. Fix this patches however you feel comfortable. Remember, the tree you're trying to apply them to now matches whatever is in ~/src/linux (which should be Linus' latest master branch).
      • You are also highly likely to run into new config options once all the patches apply. Review these options and edit the appropriate config-* files to add them in.
4. Repeat step 3 until all patches apply and all config options are answered
5. Do a local build. Fix issues until it completes.
   1. fedpkg local
      • Optionally, do a cross build of arm using scripts/cross-arm
6. Once everything builds, test the local RPMs on a machine or something.
7. Commit, push, and build
   1. fedpkg commit -c; git push; fedpkg build --nowait;

YAY FOR YOU!

Ok, so some variations to the above.

-rcX release

If you need to do a build of an upstream -rc release, the steps are mostly the same. However, we don't generate those patches ourselves and use them upstream kernel.org patches. Step 2 above is replaced with:

• Download the kernel.org -rc patch and upload it with e.g. fedpkg upload patch-4.3-rc1.xz
  • Update rcrev and gitrev in kernel.spec to match the -rcX release and 0 respectively.
  • Disable debugging options by running make release and then fixup baserelease to 1 if necessary.

Then follow the rest of the steps above. The next time you go to do a gitrev, make sure you reenable debug options with make debug

Final release

When Linus cuts a final release, we do a few things different yet again. Step 2 is now replaced with:

• Download the kernel.org linux-<version>.tar.xz tarball and upload it with e.g.
  • fedpkg upload linux-4.3.tar.xz
• In kernel.spec edit:
  • released_kernel to 1
  • rcrev to 0
  • gitrev to 0
• Generate the perf-man tarball
  • pushd ~/src/linux/; cd tools/perf/Documenation; make (remember, your git tree should be always tracking Linus' so your master branch should be at the cut final release tag.
  • tar -czvf ~/kernel/perf-man-4.3.tar.gz *.1 ; make clean
  • fedpkg upload perf-man-4.3.tar.gz
• Disable debugging options by running make release and then fixup baserelease to 1 if necessary.

Then follow the rest of the steps above. With the opening of the next merge window, make sure you reenable debug options and set released_kernel to 0.