Staying close to upstream projects

Fedora Upstream Guidelines

Fedora Project has a strong focus on not deviating from upstream as much as possible in all the different software it includes in the repository. The following is a general set of best practice guidelines on why this is a good idea, tips for sending your patches upstream, and the potential exceptions Fedora might make. The primary goal is to share the benefits of a common codebase for end users and developers while simultaneously reducing unnecessary maintenance efforts.

upstream (noun) In open source projects, the upstream of a program or set of programs is the project that develops those programs. Fedora is downstream of those projects. This term comes from the idea that water and the goods it carries float downstream and benefit those who are there to receive it.

to upstream (verb) A short-hand way of saying "push changes to the upstream project".

Why Upstream

• Common Benefit And Reduced Maintenance Burden - When Fedora carries patches that are specific to Fedora or deviations from upstream projects, those patches are not shared by every distribution. This puts the burden of maintaining those patches on Fedora, which includes keeping those patches in a functional state by rewriting or forward-porting them for every upstream release. This effort can quickly add up to be overwhelming, while not being in the spirit of sharing the benefits (as well as the effort in maintaining) free and open source software.

• Documentation - Upstream projects are documented formally (in the form of man/info pages and longer guides) and informally in many user forums or mailing lists as answers to user questions. Deviations from upstream projects can thus confuse end users when those patches cause changes in behavior that are Fedora specific and not documented properly, even where the formal documentation (such as man pages) are also patched to describe the changes.

• Translations - Maintaining translations upstream gains the advantage of established translation communities, which are most likely more experienced with the project terms. Downstream projects benefit when using those translations. In addition, it lifts the burden of hosting, and of the need to merge often from upstream to downstream and back again. To make sure translation work continues upstream, Fedora developed Transifex , which helps translators contribute directly to upstream projects. Any string changes that were introduced by patches should be maintained by the downstream community.

• Upstream Acceptance - Fedora is downstream of many thousands of software projects. Much of this software is packaged by maintainers who are not programmers or lack expertise in the language(s) the software is written in. In some cases the software has a large and complex codebase (such as the kernel or openoffice.org) and the package maintainers might not have the level of understanding in all areas compared to subsystem maintainers or upstream developers. For this and other reasons, Fedora needs the acceptance of upstream software developers. These developers may view any significant patches as a fork or refuse to take bug reports from Fedora users due to the differences in the codebase. Fedora as a project strives to be welcoming and cooperative with upstream developers as much as possible. We must avoid Fedora specific patches and any patches that are useful should be sent upstream to these developers via mailing lists, bug trackers, or direct email.

• Quality Assurance - Patches that are accepted upstream are usually reviewed or tested by many people including developers and testers. This includes testing by other GNU/Linux distributions. A deviation from the common codebase used by many is a potential chance of introducing a regression that is specific to Fedora.

• Fast incremental improvements - Staying close to upstream versions is helpful when doing version bumps for updates that bring in new features. This prevents tedious backporting of only security and bug fixes. Deviations from upstream can significantly hamper the speed of delivering improvements from new versions to end users.

• ABI or API Deviations - Patches that introduce a new application binary interface (ABI) or application program interface (API) must be especially avoided, even if the ABI/API changes are planned to land upstream. When these patches get upstream (if ever), upstream developers might introduce changes in the ABI/API during the code review process before merging the code. This could break other software in the repository that makes use of the Fedora-patched ABI/API.

• Direct End User Feedback - When users run into problems with any software that is in Fedora, they can report the problems directly upstream. By not deviating from upstream, it remains a central location for all bug reports on that software, leaving Fedora package maintainers to concentrate on good packaging instead of acting in between users and upstream issues.

Tips On Upstreaming Patches

• Make the patches generic enough to be maintained by upstream developers. Explain the need for your patches, that is, what bugs they fix or what features they add. Any references to bugzilla reports or user requests can be quite useful for the developers who receive your patches.

• Divide patches into small and independent chunks that remain functional, so they can be understood, reviewed, and accepted or rejected individually.

• If the patch introduces new strings or changes existing ones, make the changes as generic as possible.

• Fix your coding style to match the upstream project's guidelines. This might seem trivial but many upstream projects insist on following their guidelines so that their codebase looks internally self consistent and is more maintainable.

• Test your changes as throughly as you can before you send them upstream. Broken patches leave a long-lasting bad impression.

• Be patient and cooperative. If feedback is offered, discuss changes, answer questions, and provide revisions that fix any problems. Don't flame or argue unnecessarily with upstream developers. The goal is to get your patches committed upstream and not to demonstrate "who is better."

Some Examples Of Exceptions

• Severe Security Issues Or Major Bug Fixes - For any major issues, waiting on a new release from upstream for a fix can be too much of a delay. In these instances, it may be better to backport those fixes from upstream and do an update in Fedora. If you are writing a new patch, send it upstream so that Fedora shares the benefits and avoid deviations in new releases that follow. Due to differences in release schedules between upstream projects and Fedora releases, maintainers would have to keep in mind feature and development freezes in Fedora and fix problems accordingly.

• Non-free or patent encumbered software - If upstream projects include software that is non-free or has known patent issues, such software does not meet the Fedora licensing guidelines and Fedora will not include them. In many instances, such code is optional in the form of plugins that Fedora simply does not need to include in it's software repository. In other cases, it might be possible to work with upstream on making it optional or patch specific portions.

• Dead Or Unresponsive Upstream Projects - In cases where upstream projects are either dead or unresponsive, it might be acceptable to patch the software. If upstream is dead, you might want to consider sharing patches with other distributions or taking over maintenance if you have the time, skills, and interest. Be wary of maintaining software with no upstream since all the burden of maintaining the codebase as well as packaging issues are with you. If upstream is unresponsive and many distributions are deviating significantly, it might be a opportunity for a cross distribution fork (Similar to XFree86 and Xorg).

• Patches Heading Upstream - Any patches that are known to be headed upstream might be patched temporarily in Fedora for a high benefit addition of new features to end users. If this is done, maintenance effort of the patches should be low impact for the small amount of time until upstream merges the patches and does a new release.

• Distribution Integration - There are features that are critical or very good to have for Fedora as a distribution but not been significant enough for various upstream projects to accept related enhancements just yet. Use your discretion carefully when choosing to integrate any such patches since there is a trade off between upstream acceptance and Fedora integration and associated costs/benefits.

References

• http://fedoraproject.org/wiki/Objectives
• http://fedoraproject.org/wiki/PackageMaintainers/TrackingUpstream
• http://fedoraproject.org/wiki/PackageMaintainers/TrackingDownstream
• https://www.redhat.com/archives/fedora-devel-announce/2007-September/msg00019.html - No more separate kernel module packages.
• http://kernelslacker.livejournal.com/85039.html Dave Jones, Fedora kernel co-maintainer on Fedora kernel patches
• http://fedoraproject.org/wiki/Extras/GetKernelModulesUpstream
• http://kernelnewbies.org/UpstreamMerge
• http://www.kroah.com/linux/talks/ols_2002_kernel_codingstyle_talk/html/
• http://umeet.uninet.edu/umeet2002/talk/2002-12-11/linux1.txt.html
• http://kernelnewbies.org/New_Kernel_Hacking_HOWTO/Kernel_Programming_Style_Guidelines
• https://fedorahosted.org/transifex/ -- Transifex system to submit translations upstream