Flatpaks without Modules
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
Summary
Change how we build Flatpaks in Fedora to remove the dependency on modularity. Instead of using modules to rebuild Fedora packages with prefix=/app, there will be a separate build target that is used for that.
Owner
- Name: Owen Taylor
- Email: otaylor@redhat.com
Current status
- Targeted release: Fedora Linux 39
- Last updated: 2023-05-03
- FESCo issue: <will be assigned by the Wrangler>
- Tracker bug: <will be assigned by the Wrangler>
- Release notes tracker: <will be assigned by the Wrangler>
Detailed Description
There are two types of Flatpak containers - runtimes - which contain unmodified Fedora packages, and applications - which contain Fedora packages rebuilt to relocate them from /usr to /app. Currently, the rebuild process is done using modules - packages are either rebuilt in a shared flatpak-common module, or in a module specific to the application. This change replaces this with a separate build target that all rebuilds are done in.
Building
For each Fedora release, we'll have two additional targets
f39-flatpak-runtime (inherits f39), tags: f39-flatpak-runtime-build, f39-flatpak-runtime f39-app (inherits f39-flatpak-runtime), tags: f39-app-build, f39-app
The f39-flatpak-runtime target has a limited use - it's used to build packages that go into the Flatpak runtimes (standard and KDE). This would include flatpak-runtime-config, flatpak-rpm-macros, but also rebuilds of standard packages if needed to prune dependencies: currently this is done for gstreamer-plugins-good.
The f39-app target is the main target that is used for prefix=/app rebuilds. flatpak-rpm-macros is part of the build group for this target, resulting in builds that target /app. As currently, spec files can be conditionalized with '%if 0%{?flatpak}'
Any Fedora packager can rebuild a package into f39-app using 'fepdkg build --target=f39-app'. There will also be a new convenience subcommands of fedpkg 'fedpkg build-flatpak-rpms', 'fedpkg build-flatpak-rpms-local' that can be used to rebuild all missing bundled dependencies of a Flatpak, in Koji or locally.
Once a package exists in f39-app or f38-app, then [distrobuildsync] will be used to do a build into f39-app each time a standard build completes.
Updates
The current plan is that builds of RPMs will be tagged directly into f39-app and containers will be composed from the contents of f39-app with no involvement of Bodhi. Bodhi is only involved in releasing the container.
We can add CI checks on container updates to compare the rebuilt packages in the container to the latest released versions for Fedora, though it's not always easy to distinguish:
- Newer because of a RPM change that hasn't passed testing yet
- vs. Newer because a spec file was needed to get the package to rebuild with prefix=/app
Since Fedora does not embargo builds, there's no potential information leak if a change to dist-git is released via container before it is released by RPM.
Disadvantages and issues
In moving to this system, we lose the ability to take advantage of the capabilities of modularity - to use a different stream branch of a library for an application than the one shipped in F39, say. There are essentially no examples of this being used in all the Flatpaks created for Fedora over the last 5 years, probably because modularity has gained little traction within Fedora more broadly.
Some Flatpak modules strip down bundled packages by setting macros in the buildopts/rpms/macros section of the modulemd file. For example, flatpak-common has:
buildopts:
rpms:
macros: |
%_without_mingw 1
%_without_tkinter 1
This is no longer possible, so these changes need to be fed back into spec files with '%if 0%{?flatpak}' style conditionalization.
Additionally a couple of packages (evolution-data-services and tracker-miners) are set up so they can be built with an application-specific D-Bus prefix. Evolution has:
buildopts:
rpms:
macros: |
%_eds_dbus_services_prefix org.gnome.Evolution
This will need to be redone so that evolution-data-services doesn't need recompilation and the prefixing can be done by changing configuration files at container build time.
Feedback
Benefit to Fedora
The experience for Fedora developers who want to create Flatpaks of their applications will be improved:
- They no longer need to understand modularity and modulemd files - instead of a modulemd file *and* a container.yaml, they simply create a container.yaml.
- Rebuilds are automatically shared between Flatpaks, without having to get them added to the flatpak-common module.
- Because the local build experience is no longer using the poorly maintained local-build mode of MBS, it can be more efficient and easier to understand.
This should result in more Fedora Flatpaks and more up-to-date Fedora Flatpaks.
We also greatly reduce the usage of Module Build Service within Fedora. This is important because upstream maintenance is currently reduced, and it's not clear that we'll be able to sustain a deployment indefinitely.
Scope
- Proposal owners:
- Update flatpak-module-tools to be able to work without modularity
- Create changes to fedpkg to add 'rebuild-flatpak-packages' (implementation likely within flatpak-module-tools)
- Update atomic-reactor to be able to build Flatpaks without modularity
- Other developers:
- Flatpaks will need to be migrated to the new system by Flatpak packagers. While we could make the new system opt-in for F39, it's probably easiest to just require all Flatpaks to be moved over. In many cases, this should consist of just a few minor changes to container.yaml.
- Changes to fedpkg and atomic-reactor need to be reviewed, and for atomic-reactor deployed. Note Fedora is on OSBS 1, which is not a current release. Backporting changes to OSBS 1 seems more realistic than upgrading Fedora to OSBS 2 at this point.
- Release engineering: #Releng issue number
- Release engineering will need to manually create the tags/targets for F39 and update the tag maintenance scripts to do this for future releases.
- Policies and guidelines: https://docs.fedoraproject.org/en-US/flatpak/ will need to be updated once we have an initial implementation. No impact to packaging guidelines is expected.
- Trademark approval: N/A (not needed for this Change)
- Alignment with Community Initiatives: No
Upgrade/compatibility impact
The Flatpaks built will look exactly the same as before from a user perspective.
How To Test
Testing here will be done by selecting a few Flatpaks and trying to rebuild them using the new system, first locally and then on infrastructure.
The final validation will be when we mass rebuild Flatpaks for F39.
User Experience
This should be largely invisible to users. Hopefully, by simplifying the developer experience around creating Flatpaks, more Fedora Flatpaks will be available and they will be kept more up-to-date.
Dependencies
This is unrelated to other Fedora changes or to changes within packages.
Contingency Plan
- Contingency mechanism: The current infrastructure for module-based builds will be maintained, so that if we can't get this done for F39, we'll build Flatpaks as modules for F39.
- Contingency deadline: By 2023-08-01 we should have runtimes built and a number of applications or we'll plan on using modules for F39 Flatpaks.
- Blocks release? No
Documentation
N/A (not a System Wide Change)