The Fedora Legacy QA (Quality Assurance) process
Goals
The Fedora Legacy Project QA process is aimed at producing update packages which meet, to the best of our ability, the following goals.
- all packages build, install and work properly.
- we do not introduce any additional exploits, trojans or security issues.
- we provide updates for discovered security issues in a timely fashion.
- we do not infringe on copyrights or patents rights and take care to avoid legal issues.
Overview of the QA Process
The QA Process is a two step process. In the first step, package updates are created by the community and then posted for QA testing. Package submissions in the QA process are stored in the Bugzilla system ('Fedora Legacy' component). After a minimum number of other developers have checked the package against the QA Checklist below, posted their results to the Bugzilla entry, any problems found are fixed and tested again, and the package and reviews satisfy the Publish Criteria below, then the package is published to the updates-testing channel. Here, in the second step of the QA process, it must again get a minimum number of verify votes which satisfy the Publish Criteria below in order to be published to the updates channel for general public consumption.
How trustworthy is the Fedora Legacy QA Process?
The Fedora Legacy QA Process is only as trustworthy as its community members. However, we do take appropriate steps to try to make sure that the QA process, and all other Fedora Legacy resources, can be trusted. Please see the section How trustworthy are Fedora Legacy resources? at http://www.fedoralegacy.org/about/ for more information on how trust is established and enforced within the community.
How to help
First, please introduce yourself!
Being connected only through an anonymous network, it is important for us to get known to you. If you haven't done so yet, please post a self-introduction to the fedora-legacy-list mailing list. The SelfIntroduction page gives an idea of what kind of information we want to know. You may of course provide additional information about yourself if you desire. You are required to include a GnuPG key to enable us to check your signatures on packages that should be published. This step is required to help with either of the two steps of the QA process. See the PGPHowTo page for information on using GnuPG.
Next, decide how you want to help
If you are a developer, you may want to help with both steps in the QA process. But many end users will only want to participate in the second step (the updates-testing verification). If you want to participate in the first step, then continue reading about Submitting Packages and Testing packages for release to updates-testing below. If you only want to help with the second step, skip to the Testing packages for release to updates section further down this page.
The issues/packages needing different kinds of work are listed on
- http://www.netcore.fi/pekkas/buglist.html (all)
- http://www.netcore.fi/pekkas/buglist-rhl73.html
- http://www.netcore.fi/pekkas/buglist-rhl9.html
- http://www.netcore.fi/pekkas/buglist-core1.html
- http://www.netcore.fi/pekkas/buglist-fc2.html
- http://www.netcore.fi/pekkas/buglist-fc3.html
Submitting Packages
If you decide to create update packages, see QA Submit for more.
Testing packages for release to updates-testing (PUBLISH)
The goal of this is step is to check that the proposed updates, patches, etc. are correct and free from trojans before being built to updates-testing.
See QA Publish for detailed information.
Testing packages for release to updates (VERIFY)
The goal of this step is to check that the packages built seem to be working as designed. We do not have resources for extensive testing, but as we mostly use patches from already-QA'd sources like RHEL, VERIFY step can be rather straightforward.
See QA Verify for detailed information.
After testing a package
Report your test results in the Bugzilla system. QA comments in Bugzilla should be GnuPG clearsigned ( PGPHowTo ). In cases where a QA message would lead to package approval or publication, the message MUST be GnuPG clearsigned, and should contain sha1sums of the reviewed source or binary RPM packages(s). The source RPM sha1sum is mandatory in step 1 of the QA process in order to be sure that the package to be built is the same as the reviewed one. GnuPG clearsigned messages that give good advice can all be traced back to your GnuPG key and accumulate over time. This process allows new developers to gain trust through technical correctness and hard work over a period of time, eventually being able to prove to the community that the developer can be trusted.
If QA comments are submitted listing things within the package that should be fixed, the original packager should update their SRPM or respond why they feel suggestions are not valid. Be sure to increment the release number before building the new SRPMS. Upload the new SRPMS along with updated clearsigned sha1sums to your web server.
To vote that a package be moved into updates-testing, be sure to use the tag PUBLISH clearly in the message (e.g. "I vote to PUBLISH"). To vote that a package be moved from updates-testing to updates, be sure to use the tag VERIFIED clearly in the message.
Publish Criteria for updates-testing (PUBLISH)
A package may be published to updates-testing once it has at least one (preferably two) clearsigned PUBLISH QA approval from any Fedora Legacy member. However, a trusted Fedora Legacy member may veto the approval if they have a good reason to do so.
Each of the SRPMs for different OS versions require at least one PUBLISH vote to be released to updates-testing.
Publish Criteria for updates (VERIFY)
A package may be published to updates based on the following guidelines (updated in February 2006):
1. Packages will usually be released automatically in 2 weeks even if it doesn't get any VERIFY votes.
2. At package PUBLISH time, the packager and/or publisher, if they think the changes are major enough (e.g., non-QAed patches etc.), they can specify that the package should not be automatically released.
3. Negative reports block automatic publishing until the issue is resolved.
4. Positive reports can speed up automatic publishing, as follows:
- 2 VERIFY votes: released within 1 week of the first VERIFY
- all VERIFY votes: released immediately after the last VERIFY
All VERIFY votes should be clearsigned, and may come from any Fedora Legacy member.
A trusted Fedora Legacy member may veto the approval if (s)he has a good reason to do so. Package creator can also VERIFY his/her own packages.