Latest revision as of 14:12, 25 February 2022

ostree native containers (Preview) / CoreOS layering

Summary

Enhance the (rpm-)ostree stack to natively support OCI/Docker containers as a transport and delivery mechanism for operating system content.

This is the basis of https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md

In Fedora 36, this is a "preview" - significant functioning code is shipping, but we are not yet ready to finalize everything and declare that all interfaces and data formats are stable.

Owner

Name: Colin Walters

Email: walters@verbum.org

Current status

Significant code has been written and is shipping - you can try this today! You need at least rpm-ostree v2021.14 and skopeo 1.5.1

Targeted release: Fedora Linux 36
Last updated: 2022-02-25
devel thread
FESCo issue: #2704
Tracker bug: #2030707
Release notes tracker: #777

Detailed Description

Having the Fedora ecosystem (from users to release engineering) maintain tooling that operates on all three of "container images", RPMs, and OSTree updates is a maintenance burden.

This proposes that:

The ostree stack is enhanced to support encapsulating/unencapsulating ostree commits as OCI/Docker images (DONE)
rpm-ostree is updated to consume this, while still supporting all its current features (e.g. per-machine package layering) (DONE)
We ship e.g. quay.io/fedora/coreos:stable and quay.io/fedora/silverblue:36 etc. (Current status: quay.io/coreos-assembler/fcos:stable exists)
We support deriving new user custom images from these images
We enhance this tooling to support chunking

For more details, please see:

Note that significant effort has been invested in ensuring compatibility between what exists in ostree today and OCI/Docker container image "encapsulation". For example, we will continue to reuse the GPG signature infrastructure on OSTree commits that exists today - the ostree tooling knows how to verify the signature *inside* the container image. In the future, we will also likely invest in container-native signatures.

Feedback

Benefit to Fedora

Stronger focus on Docker/OCI as transport for operating system and applications
New ability to easily create derived operating system images "server side"
More benefit from e.g. work on container deltas

Scope

Proposal owners:

Lots of detailed items listed in the rpm-ostree/CoreOS docs.

Other developers: The "other" here is vague, but certainly developing this so far has needed cooperation with e.g. the containers/ organization etc.

Release engineering: https://pagure.io/releng/issue/10399

Policies and guidelines: N/A (not needed for this Change)

Trademark approval: N/A (not needed for this Change)

Alignment with Objectives: No

Upgrade/compatibility impact

Each individual edition/spin would need to choose when and how to make a cutover to containers as a transport. The Fedora OSTree repository would continue to be maintained until that is finished.

How To Test

See the examples under https://coreos.github.io/rpm-ostree/container/

User Experience

Users of rpm-ostree systems will primarily interact with container images.

Dependencies

Release engineering.

Contingency Plan

Contingency mechanism: Continue to ship updates via baseline OSTree
Blocks release? No

Documentation

Already linked above to avoid duplicating it here.

Release Notes

@@ Line 2: / Line 2: @@
 <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
-= ostree native containers / CoreOS layering =
+= ostree native containers (Preview) / CoreOS layering =
 == Summary ==
@@ Line 9: / Line 9: @@
 This is the basis of https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md
+In Fedora 36, this is a "preview" - significant functioning code is shipping, but we are not yet ready to finalize everything and declare that all interfaces and data formats are stable.
@@ Line 21: / Line 23: @@
 == Current status ==
-[[Category:ChangePageIncomplete]]
+[[Category:ChangeAcceptedF36]]
+Significant code has been written and is shipping - you can try this today!  You need at least [https://bodhi.fedoraproject.org/updates/FEDORA-2021-5e6a48669c rpm-ostree v2021.14] and [https://bodhi.fedoraproject.org/updates/FEDORA-2021-b8fee395ca skopeo 1.5.1]
 <!-- When your change proposal page is completed and ready for review and announcement -->
 <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
@@ Line 39: / Line 44: @@
 ON_QA -> change is fully code complete
 -->
-* FESCo issue: <will be assigned by the Wrangler>
+* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/KQKCFXG4CTT6IGCGWAXIX5SLHFBFCEAI/ devel thread]
-* Tracker bug: <will be assigned by the Wrangler>
+* FESCo issue: [https://pagure.io/fesco/issue/2704 #2704]
-* Release notes tracker: <will be assigned by the Wrangler>
+* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=2030707 #2030707]
+* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/777 #777]
 == Detailed Description ==
@@ Line 48: / Line 54: @@
 This proposes that:
+* The ostree stack is enhanced to support encapsulating/unencapsulating ostree commits as OCI/Docker images (DONE)
+* rpm-ostree is updated to consume this, while still supporting all its current features (e.g. per-machine package layering) (DONE)
+* We ship e.g. `quay.io/fedora/coreos:stable` and `quay.io/fedora/silverblue:36` etc.  (Current status: `quay.io/coreos-assembler/fcos:stable` exists)
+* We support '''deriving''' new user custom images from these images
+* We enhance this tooling to [https://github.com/ostreedev/ostree-rs-ext/issues/69 support chunking]
+For more details, please see:
+* [https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md CoreOS layering enhancement]
+* [https://coreos.github.io/rpm-ostree/container/ rpm-ostree container docs]
+* [https://github.com/ostreedev/ostree-rs-ext/ ostree-rs-ext project]
+Note that significant effort has been invested in ensuring compatibility between what exists in ostree today and OCI/Docker container image "encapsulation".  For example, we will continue to reuse the GPG signature infrastructure on OSTree commits that exists today - the ostree tooling knows how to verify the signature *inside* the container image.  In the future, we will also likely invest in container-native signatures.
 == Feedback ==
@@ Line 53: / Line 73: @@
 == Benefit to Fedora ==
-<!-- What is the benefit to the distribution?  Will the software we generate be improved? How will the process of creating Fedora releases be improved?
-      Be sure to include the following areas if relevant:
-      If this is a major capability update, what has changed?
-           For example: This change introduces Python 5 that runs without the Global Interpreter Lock and is fully multithreaded.
-      If this is a new functionality, what capabilities does it bring?
-           For example: This change allows package upgrades to be performed automatically and rolled-back at will.
-      Does this improve some specific package or set of packages?
-           For example: This change modifies a package to use a different language stack that reduces install size by removing dependencies.
-      Does this improve specific Spins or Editions?
-           For example: This change modifies the default install of Fedora Workstation to be more in line with the base install of Fedora Server.
-      Does this make the distribution more efficient?
-           For example: This change replaces thousands of individual %post scriptlets in packages with one script that runs at the end.
-      Is this an improvement to maintainer processes?
-           For example: Gating Fedora packages on automatic QA tests will make rawhide more stable and allow changes to be implemented more smoothly.
-      Is this an improvement targeted as specific contributors?
-           For example: Ensuring that a minimal set of tools required for contribution to Fedora are installed by default eases the onboarding of new contributors.
-     When a Change has multiple benefits, it's better to list them all.
-     Consider these Change pages from previous editions as inspiration:
+* Stronger focus on Docker/OCI as transport for operating system and applications
-     https://fedoraproject.org/wiki/Changes/Annobin (low-level and technical, invisible to users)
+* New ability to easily create derived operating system images "server side"
-     https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo (low-level, but visible to advanced users)
+* More benefit from e.g. work on container deltas
-     https://fedoraproject.org/wiki/Changes/VirtualBox_Guest_Integration (primarily a UX change)
-     https://fedoraproject.org/wiki/Changes/NoMoreAlpha (an improvement to distro processes)
-     https://fedoraproject.org/wiki/Changes/perl5.26 (major upgrade to a popular software stack, visible to users of that stack)
--->
 == Scope ==
 * Proposal owners:
-<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
-* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Lots of detailed items listed in the rpm-ostree/CoreOS docs.
-<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
+* Other developers: The "other" here is vague, but certainly developing this so far has needed cooperation with e.g. the containers/ organization etc.
-* Release engineering: [https://pagure.io/releng/issues #Releng issue number] <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Release engineering: https://pagure.io/releng/issue/10399
-<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.
-The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
-* Policies and guidelines: N/A (not needed for this Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Policies and guidelines: N/A (not needed for this Change)
-<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. Please submit a pull request with the proposed changes before submitting your Change proposal. -->
 * Trademark approval: N/A (not needed for this Change)
-<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://pagure.io/Fedora-Council/tickets/issues ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
-* Alignment with Objectives:
+* Alignment with Objectives: No
-<!-- Does your proposal align with the current Fedora Objectives: https://docs.fedoraproject.org/en-US/project/objectives/ ? It's okay if it doesn't, but it's something to consider -->
 == Upgrade/compatibility impact ==
-<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Each individual edition/spin would need to choose when and how to make a cutover to containers as a transport.  The Fedora OSTree repository would continue to be maintained until that is finished.
 == How To Test ==
-<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.
-Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
+See the examples under https://coreos.github.io/rpm-ostree/container/
-A good "how to test" should answer these four questions:
-. What special hardware / data / etc. is needed (if any)?
-. How do I prepare my system to test this change? What packages
-need to be installed, config files edited, etc.?
-. What specific actions do I perform to check that the change is
-working like it's supposed to?
-. What are the expected results of those actions?
--->
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 == User Experience ==
-<!-- If this change proposal is noticeable by users, how will their experiences change as a result?
- This section partially overlaps with the Benefit to Fedora section above. This section should be primarily about the User Experience, written in a way that does not assume deep technical knowledge. More detailed technical description should be left for the Benefit to Fedora section.
+Users of rpm-ostree systems will primarily interact with container images.
- Describe what Users will see or notice, for example:
-  - Packages are compressed more efficiently, making downloads and upgrades faster by 10%.
-  - Kerberos tickets can be renewed automatically. Users will now have to authenticate less and become more productive. Credential management improvements mean a user can start their work day with a single sign on and not have to pause for reauthentication during their entire day.
- - Libreoffice is one of the most commonly installed applications on Fedora and it is now available by default to help users "hit the ground running".
- - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
--->
 == Dependencies ==
-<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this change depends?  In other words, completion of another change owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel change)? -->
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Release engineering.
 == Contingency Plan ==
-<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
+* Contingency mechanism: Continue to ship updates via baseline OSTree  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Blocks release? No
-<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
-* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
-* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 == Documentation ==
-<!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Already linked above to avoid duplicating it here.
-N/A (not a System Wide Change)
 == Release Notes ==

Search

Changes/OstreeNativeContainer: Difference between revisions