User:Mizdebsk/FedoraMavenRepository

Fedora Maven Repository

Preamble

What is Maven Central and what should Fedora Maven repository be? (initial thoughts, obstacles)

• Maven Central primarily provides binary files, not source code
  • although it can distribute source code alongside binary JARs (artifacts), there is no guarantee that the binary JAR file was built from given source files (no trustworthy tarball->JAR mapping exists in Maven world)
  • even if provided, source code is often incomplete (Java sources are included, but other files such as build scripts or documentation are missing)
• Fedora Maven repository should, ideally, provide fully compatible Maven artifacts *without* any Fedora specific patches
  • this is to ensure that people get exactly what they expect, no incompatibilities with Maven Central
  • we can patch the artifacts on distribution level, if necessary
    • we can deploy such patched artifacts back to the Maven repository, if needed (with some Fedora-specific suffix, so they wouldn't collide with pristine ones)
• since people expect to get binary artifacts from Maven repository, we need to build them from sources
  • content review and license check is not enough in case of Java
  • to build some artifact, we need to build all its dependecies (including transitive dependencies) first
    • and we need to build all these dependencies in exact versions as upstream defines them, otherwise our artifact XYZ would be something different than artifact XYZ from Maven Central repository
      • this would mean quite a lot of extra work for us; currently we just make sure that XYZ works with what we have in distribution, we don't need to care about exact versions of dependencies
• supportability:
  • for Maven Central (and Maven repositories in general) applies:
    • once artifact is deployed into the repository, it cannot be altered, replaced or removed
    • every update, even security fixes, requires releasing new artifact version and changing all uses of that artifact to reference the new, updated version
  • this means that Fedora Maven repository would only grow in time, so supporting such ever growing repository is probably not feasible

PROPOSAL #1

This is mostly for Java developers who want to:

• write software that can be easily packaged for Linux distributions
• use our Maven artifacts with clear licensing and possibly security updates/bugfixes in their own projects

How to achieve it:

• create Maven repository
  • the goal is to have repository with clean content that can be easily used in Fedora (CentOS, ...), when needed
  • only artifacts which are actually needed by some Java application (for runtime) in still supported Fedora release will be officially supported
    • i.e. vast majority of artifacts in the repository will not be supported
• (optional) add metadata for artifacts in the repository:
  • whether artifact is vulnerable to some CVE
  • whether artifact is old and therefore it should be replaced by something else
  • what is the actual license of the artifact
  • whether artifact is suitable for usage in Fedora at all
  • it bundles content from xyz
  • it's a fork of xyz
  • ...
• (optional) create tool which will help developers to adjust their projects' configuration, so their projects will be "compatible" with Fedora Maven Repository as much as possible
  • this tool should use metadata information attached to artifacts in our repository and guide developers which dependecies they should use
  • the assumption here is that developers will actively check whether their project is compatible with Fedora Maven Repository and that they will update dependencies accordingly
• there should be an easy way of generating RPMs from artifacts in Fedora Maven Repository
  • this could be done automatically
  • we could restrict which artifacts can be added to the distribution in form of RPM (e.g.: no known security bugs, ...)
  • only RPM-ified artifacts would be supported, as long as given Fedora release is supported
    • whoever needs some particular artifact in distribution will have to maintain it (in Fedora Mave Repo)
  • only runtime dependencies of Java applications would be RPM-ified, build-time dependencies would be available through dnf maven plugin (if rebuilding from sources is needed)
    • getting rid of build-time dependencies would greatly reduce number of Java RPMs

Summary:

• pros:
  • much less Java RPMs in distribution (only runtime dependencies)
    • but this can be achieved by other means as well
• closer to how Maven world works
• friendly to Java developers who don't know anything about RPM

• cons:
  • this would initially require a *huge* amount of work
    • and non-trivial amount of continuous work
  • it touches/breaks everything that we currently have
    • convincing people that this is how Java "packaging" in Fedora should look like would be probably very challenging
  • it assumes that upstream developers will actively want to cooperate
    • the question is: is there a real demand for such repository?
  • requires a lot of storage
    • most of artifacts will have duplicate versions
    • old versions cannot be removed
    • initial storage at least 1TB, will grow with time
  • requires a dedicated CDN

PROPOSAL #2

For external developers which want to build their software for Fedora.

How to achieve it:

• periodically convert contents of RPMs into standard Maven repository
  • gererate different repos for different distros (F21, F20, etc)
  • put license files in the repo itself next to artifacts

Summary:

• pros:
  • up-to-date Maven repos with the same contents as Fedora packages
  • people can build software for Fedora on different systems, even non-Linux
  • contents can be "supported" the same way as packages in Fedora are
  • doesn't require much ongoing work, just the initial effort
    • initial effort is quite small compared to proposal #1
  • much less storage needed than in case #1
    • we would keep only 1 or 2 versions at time; older versions woul be removed
    • fedorapeople has enough free storage
• cons:
  • sort of "unstable" Maven repository, because it would change its content quite often (this applies for Fedora only)
    • but this is solvable
  • repositories tied to Fedora releases would be EOLed far too quickly

PROPOSAL #3 (problem with plugins)

We ship Maven, but it uses plugins from Maven Central (unsupported). We want users to use plugins maintained by us, if possible.

How to achieve it:

• implement and use extensions for Maven:
  • this is standard practice recommended by Maven upstream
  • Maven would use plugins provided by us, instead of downloading them from Maven Central

Summary:

• pros:
  • full Maven stack (Maven+selected plugins) coming from RPMs
  • everything there is maintained by us
  • this should be quite easy to implement
• cons:
  • we don't know which plugins will developers need, so we can provide only basic and most popular plugins

PROPOSAL #4 (installation of Maven artifacts)

Users have no "nice" way of installing Maven artifacts packaged as RPMs.

How to solve it:

• implement dnf maven plugin for:
  • easy installation of Maven artifacts (e.g.: dnf maven install commons-io:commons-io)
  • installation of build-time dependencies (e.g.: dnf maven build-dep pom.xml), so developers could then easily use XMvn to build their projects

Summary:

• pros:
  • definitely more user-friendly for people not involved in Java packaging
• cons:
  • ?

Conclusion

Implementation of Maven repository within Fedora is feasible, but maintenance of such repository seems to be more time consuming in almost every aspect than our current approach. From this reason, we would recommend to postpone this effort and wait until we see a real demand for such repository. Partial problems like usefulness of Maven without plugins can be addressed by other means, without big changes to current packaging workflow.