Talk:KojiMavenSupport

It appears that the idea is to build binaries in the buildsystem using maven instead of a chroot with rpmbuild, then dump those binaries into an srpm with a minimal spec file to install those binaries onto the system and then distribute the rpm built from that. This violates some pretty fundamental assumptions of how we package. The listed drawback that rpmbuild --rebuild will no longer be able to rebuild a package from source is a very serious change to our current policies that I think is a showstopper in and of itself. However, I think this proposal attempts to be true to some of the spirit of why we build via rpm while attempting to build via this new method so it deserves more than just a gut response. Let's try to break down some of the problems with this proposal, identify the problems it's trying to address, and work out a better solution.

High Level Goal Confusion

The first problem is that this isn't clear about what problem we're trying to solve. The "Why should we use it?" section attempts to do this but I don't know what we're after even after reading it several times. Here are several possibilities:

• Make building rpms with maven easy.
• Make building JBoss possible.
• Get rid of the current hacks.
• Do away with rpm as a build mechanism.
• Make packaging Java applications take less time.

Of these, I think that only the middle one matches up with Fedora goals. If there are hacks involved with the current build process that do not need to be there then we should get rid of them. Simplification is good.

The first two and last potential goals are possible nice-to-have's along the way towards Fedora's major goals. Yes, we want packaging to be as easy as possible. Yes, we want to take up less time on packaging so we can package more quality software. Yes, we want to be able to package things like JBoss. However, we should not compromise Fedora's goals of valuing quality over quantity, making packages that other people can look to as an example of building rpms correctly, or being a secure, maintainable distro to achieve them.

The fourth goal, doing away with rpm as a build mechanism is a non-goal of Fedora. It's practically a non-starter for this discussion. We want people to be able to take our packages and reproduce them on their own. We want them to be able to modify them and build their own derivative distros. Tying derivatives to our buildsystem rather than to the individual rpms is a heavy price that I don't think we should be prepared to pay. Fedora Infrastructure has found that koji is not a lightly deployed tool. Forcing derivative distros to have to deploy koji in order to rebuild all of our packageset is not fair to our downstreams.

Identified Problems Don't Seem to be Solved

Although the high level goals are a bit muddy, the "Why should we use it?" section does talk about the problems that keep us from reaching the potential goals that I outlined.

1. Maven presupposes that we'll have a repository of java packages inside of a Maven repo.
2. Maven builds want to pull in multiple versions of a package.
3. The current Fedora Java Guidelines lead to fragile dependencies between packages.
4. The current Fedora Guidelines are complex
5. Some application systems (such as JBoss) have a lot of dependencies that need to be built. This is a complex process when building independent rpms.

Except for #1, none of these problems will be solved by having a new koji backend using maven. These are issues where policy is conflicting with convenience. If we had a maven-based buildsystem, the same policies as we have currently would continue to cause these issues to occur. Let's take them in order:

2) Multiple versions. Currently Fedora discourages multiple versions but doesn't ban them. I believe that there is policy at the FESCo level surrounding multiple versions (which is why we don't currently have multiple versions of python, for instance). The problem with maintaining multiple versions is that most often, upstream only cares about one or two versions of their software. If we are packaging more versions of the software to satisfy dependencies than upstream supports, then we become responsible for the extra versions. If a bugfix or security fix is needed, then we become responsible for generating that for all the versions that we maintain. This type of backporting is counter to Fedora's goal of being a forward moving distro. Normally we work to port applications forward to new versions of libraries to keep the number of libraries to a minimum and soease our overall support burden over time. Having a new, maven-based build system won't change this policy. If FPC or FESCo or simply reviewers consdering these issues won't approve all the versions you think you need it doesn't matter whether the buildsystem is rpmbuild or maven based.

3) Guidelines are fragile. The Fedora Packaging Committee has taken the Guidelines that Java Developers have issued for creating Java Packages with minor changes. If they are fragile, then the Guidelines need to be fixed to be more robust. Remember that the Guidelines aren't going to change merely because we have a new buildsystem. We went from plague to koji without the Guidelines making one koji-specific change.

4) The Guidelines are complex. Once again, fix the Guidelines. If there's resistance to changing something about the Guidelines switching buildsystems isn't going to make it any better, the requirements are still going to be the same.

5) Bundled libraries are prohibited by the Guidelines. This is not going to change just because we have a new buidsystem. You mention that "Koji-Maven would enable us to build JBoss from source using the upstream build process" but that's not true. If JBoss is relying on bundled libraries, it will have to be split up whether the build process goes through a maven-based koji or an rpmbuild-based koji. If you want this to change, you need to get the Guidelines changed. And a word -- Lots of people attempt to get an exception to this Guideline. That's the wrong way to pursue change. Show that the Guideline is either not beneficial to the distribution or that the costs of following the Guideline warrant coming up with other ways to mitigate the problems the Guideline is fixing. This should be something that can apply to all packages in the distro.

Additional Costs

• Complexity. We'd have to build and maintain two backends to koji instead of

one.

• Content delivery to end users becomes more complex. Whereas we currently

only have to explain the process of downloading and building from srpms to users and customers who want to rebuild our rpms with only minor changes or verify source, CFLAGS, etc, or only run their own binaries with this change we would need to educate them on both rpmbuild and maven.

• "Self-hosting" is less meaningful. Right now, every rpm we build can be

built in a straightforward manner on Fedora. You satisfy the BuildRequires and run rpmbuild --rebuild. The necessary tools are in Fedora and pushing BuildRequires into the system is a task that has many tools to help out. (mock, yumbuilddep, etc). With this proposal, we would create an environment where our builds would not be done on pure Fedora anymore. The same content might be available in rpm format but the build environment would be different since we're building using a maven repository whereas the user is building with rpms installed locally.

• Our obligations under the GPL become a little less clear. When we talk

about the distro as a whole being licensed under the GPL we are promising to give out source to rebuild the distro. This has always meant srpms for us which not only give the user the source for the applications themselves, but the source for recreating the exact binary rpm that we used. With this proposal, the srpm no longer does that. We may still be within our legal rights (have to talk to a lawyer about that) but this is one of the things we have blasted other distributions, and upstreams that distribute binary rpms and source tarballs but not srpms/spec files for doing.

• Packagers and reviewers will have to learn about both maven and rpm spec

files in order to use this. Building a package locally as part of the review process will have to be worked out -- this proposal isn't detailed enough to tell me what the equivalent of mock SRPM would be for java packages. there's an increase in work and knowledge here that counters any decrease that occurs from having simpler rpm spec files.

Counter Proposal

Instead of making a maven repo in the build system at build time, why don't we make a maven repo on the local system at install time? For Fedora, have every jar be installed to some location by rpm. Then the rpm %post script runs to regenerate whatever metadata is needed to make it a maven repo. If maven is really the de facto standard for building Java jars, this will help not only us when we build (because our srpms will just point at the local maven repo that's created from all the java rpms that are installed in the chroot) but also the developers on Fedora systems who can now point their toolset at the maven repo that gets built locally.

--abadger1999 05:58, 26 September 2009 (UTC)

Response

Some good points have been brought up, but I think there's a general misunderstanding of how Maven does things, why that's largely incompatible with rpmbuild, and what this proposal is trying to accomplish. Questions and concerns are addressed below individually.

High Level Goal Confusion

The proposal is not about quantity vs. quality. It's about making it possible to build Java packages in a reproduceable way while being compatible with upstream, and packaging those builds as rpms. It's about making it possible for more high-quality open-source software to be part of the Fedora ecosystem in a maintainable way. I think that is very much in line with the goals of the Fedora project.

This proposal isn't trying to "do away with rpm as a build mechanism" at all. It's trying to introduce an alternate build mechanism for packages whose build process doesn't fit well into the way rpmbuild works. The vast majority of packages would be unaffected. But for packages where the overhead of building via rpmbuild prevents their inclusion in Fedora, they would have an alternative.

Using this system does not tie derivative distros to Koji any more than using rpmbuild does. All of the source code would be included in the srpms, and the Maven repos managed by Koji would be publicly available. To rebuild the jar files we're distributing, end users or derivative distros would simply have to unzip the source .zip file included in the srpm, configure Maven to point to either a Koji-managed Maven repo or another publicly-available upstream repo (like repo1.maven.org) and run /usr/bin/mvn deploy. This proposal is simply a wrapper around using Maven as a build tool, in the same way Koji is a wrapper around using mock as a build tool.

Identified Problems Don't Seem to be Solved

2) There is no way in rpmbuild to BuildRequires two different versions of the same package, which is essentially what Maven builds require. Because a single Maven project may consist of a number of subprojects, and each of those subprojects has their own dependency list, it is common for different versions to get pulled in to the same build environment. This is a fundamental assumption of Maven. For example, see the repo for plexus. Different subprojects of a single build may need to depend on different versions. These are only text files with shared configuration data, so there is no runtime component and no need to install them on a destination system, but they need to be accessible at build time. There is no way to satisfy this requirement in the context of rpmbuild.

3) It's not the guidelines that are the problem, it is the technical implementation of building Java packages using Maven within rpmbuild that is fragile. Fedora carries local patches to Maven that have been rejected by upstream to force it to use locally-installed jars instead of a network-accessible Maven repo. It overrides the dependency resolution in Maven, so packages get built against versions other than what they specify in their .pom files, potentially leading to broken builds or incorrect runtime behavior. It requires the duplication of dependency information maintained in the .pom file into the specfile. It requires the creation and maintenance of .xsl and .xml fragments that are specific to only the Fedora Java build environment. All of this dramatically increases the maintenance burden of Java packages in Fedora. The build system is completely different than the Maven system used by upstream projects, and very complex because it has to work around the different assumptions made by rpmbuild and Maven. Because the current Java build system is unfamiliar to upstream Java developers and even experienced non-Java package maintainers, the pool of people willing and able to maintain Java packages in Fedora is significantly reduced, and Java support in Fedora suffers as a result.

4) See 3.

5) The proposal doesn't say anything about bundled libraries, and has no intention of bundling pre-built binaries. It addresses the issue of JBoss having a lot of dependencies by lowering the barrier to entry of getting those dependencies built from source in the build system and included in Fedora. Adding a new Java package to the current system is complex and difficult, requires in-depth knowledge of both rpmbuild and Maven, and is really only fully understood by the people who designed the current system. It makes adding a project like JBoss with a lot of dependencies a daunting and untenable task. By simplifying the build process (making it more like the upstream build process used by Java developers) and the packaging process (leaving the build logic in the .pom files where it is maintained by the upstream developers, and using rpm as the packaging format to bundle the jars built by Maven) it becomes much easier and quicker to get a new Java project building in Koji, and makes packaging a large project like JBoss feasible.

Additional Costs

• Complexity. We'd have to build and maintain two backends to koji instead of one.
  • The two build processes would be completely interoperable. Koji rpms would be scanned for .pom and .jar files and populate the Maven repo, and Maven wrapper-rpms would be made available in the yum repos used by Koji. It would operate as a single build system.

• Content delivery to end users becomes more complex.
  • Rebuilding the .jar files from a checkout of the upstream source code (included in the srpm) is something anyone familiar with Java development should be more than capable of. We can even have the %build section of the wrapper rpms unpack the source .zip file and print a message saying "point your Maven settings to the Fedora Maven repo at http://maven.fedoraproject.org/... and run 'mvn deploy' to rebuild the .jar files". This is no more complex than setting up mock to point to a suitable yum repo and running 'mock --rebuild'.

• "Self-hosting" is less meaningful.
  • This is true, but I don't think in practice it makes it any more difficult to rebuild the code. Most people already use mock and an external yum repo (often hosted on Fedora infrastructure) to satisfy BuildRequires and do a rebuild. Rebuilding Maven packages would likewise be referencing the publicly-available Koji-managed Maven repos, which we could also mirror, or they could use other public Maven repos which host many of the same projects. This actually makes understanding the build process for Java packages much easier for the average user than it is now, and much closer to how upstream does their builds. There is no assumption that all dependencies are available locally when doing a Maven build, so self-hosting really isn't relevant in the Java world.

• Our obligations under the GPL become a little less clear.
  • We are still giving out the complete source code (in .zip format) in the srpm. The tools to build it (Maven) are available on the local system, and all of the dependencies (the Koji-managed Maven repo) are available publicly. This seems like exactly the same situation we have with our current rpm builds. We don't include *all* build tools or dependencies on all media, we make them available over the network for people who want them. It would work the same with Maven builds.

• Packagers and reviewers will have to learn about both maven and rpm spec files in order to use this.
  • Packagers *already* need to know both Maven and rpm specfiles in-depth to use the current Java build system. Even worse, they need to know about the non-standard, upstream-rejected extensions to Maven in Fedora, and a complex system of macros and xsl/xml in rpm. A primary goal of this proposal is to simplify the Java packaging process dramatically. The build works just like it does upstream, so Java developers are familiar with it and can debug problems more easily. And the specfile fragments are very simple and could be reused across the majority of packages with little or no modification, making packaging as rpms very simple even for those not accustomed to rpm packaging. This should lower the barrier to entry for a lot of Java packages and enable participation in the Fedora community by a large number of open-source Java developers.

Counter Proposal

The counter proposal does not handle the multiple version problem described above. A fundamental assumption of Maven builds is that they have access to a repo containing every version of every project built previously. There is no way to duplicate this using rpms installed on a local system without patching Maven and resorting to the kind of pom-translation and version-redirection tricks that the current Java build process uses, and results in significant complexity and maintenance burden. The best way to avoid this is to not force Maven to operate in an environment it was never designed for. Separating out the build tool (Maven) from the packaging tool (rpm) allows us to have a more maintainable process, and open the door to including a huge amount of high-quality, open-source Java software in Fedora.

-- Mike Bonnet, (mikeb in FAS, mbonnet on freenode), 2009-10-02 18:53 UTC