From Fedora Project Wiki
No edit summary
mNo edit summary
Line 4: Line 4:


== Summary ==
== Summary ==
Fedora distributed as bootable container ships via [https://quay.io/repository/fedora/fedora-bootc?tab=tags OCI registry]. Installation is typically done by conversion into a VM image or ISO installer via [https://osbuild.org/ osbuild] (image builder), however, booting from network is a useful workflow for bare-metal fleet deployments. Required files to perform such installation are not available in the OCI repository that could be fetched from registry in a similar manner as the bootable container.
Fedora distributed as bootable container ships via [https://quay.io/repository/fedora/fedora-bootc?tab=tags OCI registry]. Installation is typically done by conversion into a VM image or ISO installer via [https://osbuild.org/ osbuild] (image builder), however, booting from network is a useful workflow for bare-metal fleet deployments. Required files to perform such installation are not available in the OCI repository that could be fetched from registry in a similar manner as the bootable container.


As of today, files are only available in the Fedora RPM repository and the installation workflow would be cumbersome to find appropriate RPM repo version and extract needed files instead of fetching all the needed assets from the registry only.
As of today, files are only available in the Fedora RPM repository and the installation workflow would be cumbersome to find appropriate RPM repo version and extract needed files instead of fetching all the needed assets from the registry only. The change introduces a new OCI repository with the files in question for each Fedora stable version.


== Owner ==
== Owner ==
Line 17: Line 18:
== Current status ==
== Current status ==
[[Category:ChangeReadyForWrangler]]
[[Category:ChangeReadyForWrangler]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- The Wrangler announces the Change to the devel-announce list and changes the category to Category:ChangeAnnounced (no action required) -->
<!-- After review, the Wrangler will move your page to Category:ChangeReadyForFesco... if it still needs more work it will move back to Category:ChangePageIncomplete-->
[[Category:SelfContainedChange]]
[[Category:SelfContainedChange]]


Line 46: Line 42:
Some files are distributed unsigned in the `images/` directory, others are signed and need to be extracted from RPM packages. A complete ISO "netboot" image is also available for network installations, the image can be customized using `mkksiso` tool found in Fedora.
Some files are distributed unsigned in the `images/` directory, others are signed and need to be extracted from RPM packages. A complete ISO "netboot" image is also available for network installations, the image can be customized using `mkksiso` tool found in Fedora.


The main goal of this change is to start publishing the mentioned files as [https://oras.land/docs/concepts/artifact/ OCI artifacts] for each Fedora version and architecture. Buildah/Podman will be used for creating such manifest and [https://gitlab.com/fedora/bootc/netboot/netboot/-/blob/main/push.sh?ref_type=heads pushing it] to OCI registry.
The main goal of this change is to start publishing the mentioned files as [https://oras.land/docs/concepts/artifact/ OCI artifacts] for each Fedora version and architecture. Buildah/Podman will be used for creating such manifest and [https://gitlab.com/fedora/bootc/netboot/netboot/-/blob/main/push.sh?ref_type=heads pushing it] to OCI registry and the process will be integrated into current or upcoming (Konflux) release processes.


There is currently no support for downloading OCI artifacts with podman but the feature is currently being discussed and worked on upstream. However, Fedora contains `golang-oras` tool which understands the OCI artifact format. This tool can already be used by Fedora users to consume the content:
There is currently no support for downloading OCI artifacts with podman but the feature is currently being discussed and worked on upstream. However, Fedora contains `golang-oras` tool which understands the OCI artifact format. This tool can already be used by Fedora users to consume the content:
Line 63: Line 59:
</pre>
</pre>


There is a [https://github.com/pulp/netboot-oci-specs/blob/main/netboot-oci-artifacts.md manifest specification] describing kickstart files stored as OCI artifacts.
Alternatively, the content can be downloaded via `skopeo` tool with [https://github.com/theforeman/nboci-files/blob/main/artifact-pull.py some scripting] involving file renaming.
 
The proposed repository for the content is: `quay.io/fedora/kickstart-artifacts` and tag convention will be `N` where N is Fedora version with manifest index for all supported architectures pointing to tags in the form of `N-arch`. Only stable and N-1 Fedora versions will be kept for storage reasons and old artifacts will be regularly removed and garbage collected. For more info, read [https://github.com/pulp/netboot-oci-specs/blob/main/netboot-oci-artifacts.md manifest specification].


== Benefit to Fedora ==
== Benefit to Fedora ==


The change solves the situation for Fedora bootable containers users who currently need to find matching Fedora RPM repositories and use various tools like `curl` or `rpm2cpio` and `cpio` to download required files.
The change solves the situation for Fedora bootable containers users who currently need to find matching Fedora RPM repositories and use various tools like `curl` or `rpm2cpio` and `cpio` to download required files. This will significantly simplify provisioning workflows of Fedora systems en-masse via automation tools like Ansible or Foreman. All files will be also signed by Fedora GPG keys for increased security.


Users of regular (RPM) Fedora spin will benefit as well since bare-metal provisioning workflows, scripts or tools can be further simplified. All the content will be also signed by GPG which is not the case for some (executable) files today.
Users of regular (RPM) Fedora spin will benefit as well since bare-metal provisioning workflows, scripts or tools can be further simplified. Additionally, many provisioning systems (Beaker, Foreman) use one shim/grub for installing all OS versions which does not work reliably when SecureBoot is turned on. Published files can be easily downloaded for each OS version.


Using OCI artifacts enables us to distribute the files through the registry under e.g. `quay.io/fedora/kickstart-artifacts`.
The newly published content is planned to be integrated with other open source projects: Foreman, Pulp and Ansible. This is out of scope for this change.


== Scope ==
== Scope ==
Line 77: Line 75:
* Proposal owners: prepare CI/CD pipeline for fully automated build and push of kickstart artifacts, integrate the published repositories with related open-source project workflows [https://theforeman.org/ Foreman] and [https://pulpproject.org/ Pulp]
* Proposal owners: prepare CI/CD pipeline for fully automated build and push of kickstart artifacts, integrate the published repositories with related open-source project workflows [https://theforeman.org/ Foreman] and [https://pulpproject.org/ Pulp]


* Release engineering: create new repository in fedora namespace [https://pagure.io/fedora-infrastructure/issue/12152 #12152]
* Release engineering: create new repository in fedora namespace [https://pagure.io/fedora-infrastructure/issue/12152 #12152] and assistance with integrating the new pipeline into the Fedora workflow


== Documentation ==
== Documentation ==


The newly created repository will be features in documentation of several upstream projects that will make use of it:
The newly created repository will be features in documentation of several upstream projects that will make use of it:

Revision as of 10:52, 11 September 2024

Distributing Kickstart Files as OCI Artifacts

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Fedora distributed as bootable container ships via OCI registry. Installation is typically done by conversion into a VM image or ISO installer via osbuild (image builder), however, booting from network is a useful workflow for bare-metal fleet deployments. Required files to perform such installation are not available in the OCI repository that could be fetched from registry in a similar manner as the bootable container.

As of today, files are only available in the Fedora RPM repository and the installation workflow would be cumbersome to find appropriate RPM repo version and extract needed files instead of fetching all the needed assets from the registry only. The change introduces a new OCI repository with the files in question for each Fedora stable version.

Owner

Current status

  • Targeted release: Fedora Linux 42
  • Last updated: 2024-09-11
  • [Announced]
  • [<will be assigned by the Wrangler> Discussion thread]
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

Fedora bootable container is shipped via OCI registries without any supplementary files for automated kickstart installations. The files needed for this workflow are typically: bootloader, anaconda kernel, initramdisk and anaconda main image. These files can be found in regular Fedora RPM repository, for example in case of x86_64 architecture:

Some files are distributed unsigned in the images/ directory, others are signed and need to be extracted from RPM packages. A complete ISO "netboot" image is also available for network installations, the image can be customized using mkksiso tool found in Fedora.

The main goal of this change is to start publishing the mentioned files as OCI artifacts for each Fedora version and architecture. Buildah/Podman will be used for creating such manifest and pushing it to OCI registry and the process will be integrated into current or upcoming (Konflux) release processes.

There is currently no support for downloading OCI artifacts with podman but the feature is currently being discussed and worked on upstream. However, Fedora contains golang-oras tool which understands the OCI artifact format. This tool can already be used by Fedora users to consume the content:

$ oras pull quay.io/lzapletal/fedora-bootfiles:40-amd64
Downloaded  80c3fe2ae106 boot.iso
Downloaded  a3b7052d7b2f grubx64.efi
Downloaded  fff4b2feeef3 pxelinux.0
Downloaded  4773d74d87c2 shimx64.efi
Downloaded  09cf5df01619 vmlinuz
Downloaded  8ea1dd040e97 initrd.img
Restored    80c3fe2ae106 install.img
Pulled quay.io/lzapletal/fedora-bootfiles:40-amd64
Digest: sha256:0306e10fd556e12ce8c3674150bceb88c0917b74b63c37eecc17070b3b30003b

Alternatively, the content can be downloaded via skopeo tool with some scripting involving file renaming.

The proposed repository for the content is: quay.io/fedora/kickstart-artifacts and tag convention will be N where N is Fedora version with manifest index for all supported architectures pointing to tags in the form of N-arch. Only stable and N-1 Fedora versions will be kept for storage reasons and old artifacts will be regularly removed and garbage collected. For more info, read manifest specification.

Benefit to Fedora

The change solves the situation for Fedora bootable containers users who currently need to find matching Fedora RPM repositories and use various tools like curl or rpm2cpio and cpio to download required files. This will significantly simplify provisioning workflows of Fedora systems en-masse via automation tools like Ansible or Foreman. All files will be also signed by Fedora GPG keys for increased security.

Users of regular (RPM) Fedora spin will benefit as well since bare-metal provisioning workflows, scripts or tools can be further simplified. Additionally, many provisioning systems (Beaker, Foreman) use one shim/grub for installing all OS versions which does not work reliably when SecureBoot is turned on. Published files can be easily downloaded for each OS version.

The newly published content is planned to be integrated with other open source projects: Foreman, Pulp and Ansible. This is out of scope for this change.

Scope

  • Proposal owners: prepare CI/CD pipeline for fully automated build and push of kickstart artifacts, integrate the published repositories with related open-source project workflows Foreman and Pulp
  • Release engineering: create new repository in fedora namespace #12152 and assistance with integrating the new pipeline into the Fedora workflow

Documentation

The newly created repository will be features in documentation of several upstream projects that will make use of it:

  • osbuild
  • foreman
  •  pulp

Release Notes

TBD