Basic Idea
Flatpak (formerly xdg-app) is a way of distributing and running applications with more isolation from the system than a traditional packaging format such as RPM. Dependencies for a flatpak are either distributed as part of the runtime or are bundled into the application itself. These - the runtime and the application - are the two types of artifacts we want to build for Fedora.
- runtime
- A runtime contains files that multiple applications depend on. There can be any number of runtimes installed on a system: for example, there might be both a Fedora 23 and a Fedora 24 runtime installed. A runtime looks like a mini-system image with libraries in /usr/lib, data files in /usr/share, binaries in /usr/bin and so forth.
- application
- The application contains both the application itself and libraries it depends on that are not part of the runtime. All files in the application are under /app.
Runtimes and applications are typically distributed as ostree repositories - but also can be stored as a single-file bundle. Currently a bundle is a ostree delta, but using the OCI container format instead is being investigated.
When a flatpak is run, an isolated environment is created with the runtime mounted on /usr and the application mounted on /app. Depending on sandboxing, the environment might have full access to the user's home directory, or might only access to a restricted set of D-Bus portals.
We generally want the runtimes and applications to be built out of the same set of sources as the rest of Fedora: when we do a security update for a library, we want to know what applications or runtimes are affected and need to be rebuilt. We don't want to create an entirely new way for sources to get into Fedora bypassing the normal Fedora procedures.
Runtimes can simply be built out of normal Fedora RPMS in the same way as we might compose an ISO or a container image, but to make an application out of RPMs requires rebuilding them with the /app prefix. The most obvious way of doing this would be a separate build target, but the downside of this is that we've now forked the build process. Say we build the 'eog' image viewer as a flatpak - so we rebuild it in the special build target. It requires the 'exempi' library for reading image metadata, so we rebuild that as well. Later - the exempi maintainer updates exempi - somehow the build in the other build target has to be rebuilt in sync, or the eog bundle will contain a stale version of exempi.
What if we could consider "x86_64 but rebuilt with a /app prefix" as a separate architecture, so that whenever exempi is rebuilt, koji rebuilds that version as well? This led to the idea of "architecture variants", where a package can be configured to build not just x86_64, but also x86_64~app. See https://pagure.io/fork/otaylor/koji/blob/xdg-app/f/docs/arch_variants.rst for more details.
Packager Workflow
- A packager identifies a package as a candidate for creating a flatpak for.
- They mark the application to be built as a flatpak (koji set-pkg-variants 'MAIN flatpak' f25 eog.)
- They bump and rebuild the package
- They try and build the flatpak (koji flatpak-build f25 eog eog.json. Extra information about the app - like what resources it wants access to - is included in a JSON file that could also be checked into source control.
- If that fails because of missing library dependencies, they add the flatpak variant to these dependencies and rebuild them, and repeat the flatpak build.
Phase 1 - manual enablement
- Support in koji for architecture variants
- Add 'arch_variants' to the tag_packages schema.
- Add support in the hub to understand variants at the necessary places
- Add support in the builder to fire off builds for configured architecture variants, and when building for a architecture variant, install '<variant>-build' rather than 'build'
- A way to set the variants for a packages - perhaps 'koji set-pkg-variants' similar to 'koji set-pkg-arches'.
- A koji plugin to add ‘koji flatpak-build [--runtime] NVR METAINFO.JSON’ which takes an existing package that has been built for the ~app variants, and converts the RPM and its dependencies into a flatpak bundle (either existing ostree delta or future OCI image).
A branch of Koji with prototypes of these changes can be found at: https://pagure.io/fork/otaylor/koji/branch/xdg-app. flatpak-build is done as a built-in rather than a plugin as a first pass for development convenience.
Enhancement: autoimport into a ostree repository
- When new app/runtime bundles are built, automatically import them into a ostree repository. (Or testing/release repositories based upon how the source rpms are tagged) Conceivably this could be handled by a new separate service that listens on the message bus, rather than trying to extend koji.
Enhancement: autorebuild of flatpaks
- When packages are rebuilt that are part of flatpak applications and runtimes, automatically trigger rebuilds of the applications/runtimes. This would require tracking the packages in each application or runtime. This might be handled by the same daemon as above.
Enhancement: generalize this to server containers
- The basic idea of taking RPMs and converting the rpm and dependencies into a container via rpm-ostree is applicable to containers as well. The autorebuild and autoimport into ostree steps would be useful for this as well (but also probably auto-push to a docker registry.) One challenge here is that while an RPM of a desktop application is “self-describing” - it contains the .desktop and appstream files that specify what the application is and how to run it, turning an RPM of, say, etcd into a useful container probably requires extra metadata and configuration files.
Enhancement: merge in ~flatpak architecture variants to an existing build
- It might be nice to avoid the bump+rebuild, as envisioned by the mergeScratch RPC call that was added https://lists.fedoraproject.org/pipermail/buildsys/2015-April/004628.html
Enhancement: auto-rebuilding dependencies
- Tooling so that when someone wants to build an application that has not previously been built for ~flatpak, all the build dependencies are queued for adding to the ~flatpak architecture variants and rebuilt in the right order.