Fedora 21 Final Release Criteria

Final Objectives

The objective of the Final release is to:

1. Provide a polished final release suitable for meeting the needs of our Target Audience

Final Release Requirements

In order to be released to the general public, a compose must meet all of the following criteria. This is intended to make the decision process as clear and straightforward as possible. Mostly met items are incomplete until they are met. Optional and nice to have items should not be included in this list.

There may be times where a requirement is unmet only in a particular configuration, such as with some keyboard layouts but not others, or if a particular character is used in a username, password or passphrase. In such cases, the release team should use their judgement and refer to precedent to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by the issue, the severity of the case when the issue is encountered, and the ease or otherwise with which the issue can be avoided by both informed and uninformed users.

The term release-blocking desktops means all the desktop environments in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking desktops for x86_64 is GNOME and KDE, and for aarch64 is GNOME. Note that bugs in desktops that are not part of this set which would infringe these criteria automatically qualify for freeze exception status, according to the freeze exception bug process.

The term release-blocking images means all the images in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking images includes the images defined by the three Editions - Server, Workstation and IoT - in their product requirement documents and/or technical specifications, the Everything network install image, key Cloud images, and the KDE live image. The canonical list of release-blocking images for Fedora 42 is on this page.

Process requirements

Beta criteria met

All Fedora 21 Beta Release Criteria must be met.

Final blockers CLOSED

All bugs blocking the Final tracker must be CLOSED.

Installer requirements

Media consistency verification

Validation of install media must work correctly for all release-blocking images.

Details

This means that the installer's mechanism for verifying that the install medium is intact must complete successfully if the medium is correctly written and return a legible failure message if it is not.

References

• Proposed 2012-02-01, implemented 2012-02-20
• Re-worded 2012-06-29, extended 2012-10-24, and re-written again for major Fedora 19 criteria revision
• Test case: QA:Testcase_Mediakit_ISO_Checksums (incomplete)

Live image persistent overlays

The release-blocking live images must properly support mounting and using a persistent storage overlay for the entire system and/or one for the /home partition.

Writing not covered

This criterion does not cover the writing of the persistent overlay, as that stage is likely to be done using a stable released Fedora or other operating system and hence irrelevant to the release validation process.

References

• Proposed 2012-06-08, implemented 2012-06-29
• Test case: missing

Package and installer sources

The installer must be able to use all supported local and remote package and installer sources.

References

• Part of initial Fedora 13 criteria revision
• Test cases:
  • QA:Testcase_install_repository_DVD_default
  • QA:Testcase_install_repository_Live_Image
  • QA:Testcase_Boot_Methods_Pxeboot
  • QA:Testcase_install_repository_Mirrorlist_default
  • QA:Testcase_install_repository_Mirrorlist_graphical
  • QA:Testcase_install_repository_HTTP/FTP_graphical
  • QA:Testcase_install_repository_HTTP/FTP_variation
  • QA:Testcase_install_repository_NFS_graphical
  • QA:Testcase_install_repository_NFS_variation
  • QA:Testcase_install_repository_NFSISO_variation
  • QA:Testcase_install_repository_Hard_drive_variation

Network attached storage

The installer must be able to detect (if possible) and install to supported network-attached storage devices.

Supported types

Supported network-attached storage types include iSCSI, Fibre Channel and Fibre Channel over Ethernet (FCoE).

References

• iSCSI was listed along with local storage interfaces as part of initial Fedora 13 criteria revision
• Separate 'network-attached storage' criterion was proposed as part of a wider revision 2011-06-23, implemented 2011-07-06
• Test cases:
  • QA:Testcase_install_to_iSCSI_no_authentication
  • QA:Testcase_install_to_FCoE_target

Installation interfaces

The installer must be able to complete an installation using all supported interfaces.

References

• Proposed 2012-02-03, implemented 2012-02-13
• Test case: missing (all interfaces except cmdline are covered at Alpha and Beta, cmdline is missing)

Disk layouts

The installer must be able to create and install to any workable partition layout using any file system and/or container format combination offered in a default installer configuration.

Wait, what?

Yeah, we know. This is a huge catch-all criterion and it's subject to a lot of on-the-fly interpretation. Broadly what it's 'meant to mean' is that you should be able to do anything sane that the Installation Destination spoke attempts to let you do, without the installer exploding or failing. We are trying to write more specific criteria covering this area, but it's not easy. Patches welcome, as the kids say...

Default installer configuration?

The default installer configuration clause specifically excludes options that only appear in the installer if you do something special to trigger them. There have been times when the installer has implemented experimental support for some new filesystem by showing it if you pass a special kernel parameter, for instance: the purpose of this clause is explicitly not to cover cases like that.

References

• Part of initial Fedora 13 criteria revision
• Slightly contracted and abstracted for major Fedora 19 criteria revision
• Test cases:
  • QA:Testcase_anaconda_btrfs_rootfs_on_disk_partition
  • QA:Testcase_anaconda_ext3_rootfs_on_disk_partition
  • QA:Testcase_anaconda_xfs_rootfs_on_disk_partition
  • Plus all the Alpha and Beta paritioning test cases

Bootloader disk selection

The installer must allow the user to choose which disk the system bootloader will be installed to, and to choose not to install one at all.

References

• Added as part of major Fedora 19 criteria revision
• Test case: missing

Storage volume resize

Any installer mechanism for resizing storage volumes must correctly attempt the requested operation.

What does that cover?

This means that if the installer offers mechanisms for resizing storage volumes, then it must run the appropriate resizing tool with the appropriate parameters for the resize the user chooses. The reason it's worded this way is we specifically don't want to cover cases where the requested resize operation then fails for some reason - dirtily unmounted or over-fragmented partition, for instance. We only want to cover the case that the installer's resize code itself is badly broken.

Corruption-causing bugs covered in another criterion

This criterion does not necessarily cover all cases of bugs in resize mechanisms which may cause some kind of data corruption, but any such bug would be covered by the general data corruption criterion.

References

• Proposed 2013-07-03, implemented 2013-07-10
• Test case: QA:Testcase_Anaconda_autopart_(shrink)_install (partial)

Windows dual boot

The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.

Clean Windows installation?

This is a get-out clause for cases where there's a bug caused by some weirdness or abnormality in the Windows installation; we can't plausibly undertake to support every possible Windows deployment. The expected scenario is a cleanly installed or OEM-deployed Windows installation. Issues caused by recovery or 'system' partitions may not be considered to violate this criterion, depending on the specific circumstances. This criterion is considered to cover both BIOS and UEFI cases.

References

• Part of initial Fedora 13 criteria revision
• Simplified for major Fedora 19 criteria revision to drop unnecessary 'no bootloader' clause
• [Adjusted https://lists.fedoraproject.org/pipermail/test/2014-October/123073.html 2014-10-02] to require Fedora bootloader to boot Windows even in UEFI case
• Test case: QA:Testcase_dualboot_with_windows

OS X dual boot

The installer must be able to install into free space alongside an existing OS X installation, install and configure a bootloader that will boot Fedora; if the boot menu presents OS X entries, they must boot OS X. Installing Fedora must not inhibit the system's ability to boot OS X from the UEFI boot manager.

References

• Proposed 2014-08-24
• Various parties discussed the proposal and provided alternative wordings in the subsequent discussion
• Near-final form drafted by Chris Murphy 2014-09-09
• Requirement to preserve UEFI OS X boot capability proposed by Adam Williamson 2014-09-29
• Final form implemented 2014-10-06
• Test case: QA:Testcase_dualboot_with_OSX

Update image

The installer must be able to use an installer update image retrieved from removable media or a remote package source.

Supplementary to Alpha

Note the corresponding Alpha criterion that requires update image sourcing via HTTP to work.

References

• Proposed 2012-02-20, implemented as Beta criterion 2012-02-27, moved to Final 2012-04-04
• Test cases: QA:Testcase_Anaconda_updates.img_via_installation_source, QA:Testcase_Anaconda_updates.img_via_local_media

Installer translations

The installer must correctly display all sufficiently complete translations available for use.

Substantial breakage only

One or two translated strings not being displayed will not usually constitute a violation of this criterion (unless the ones missing are very important strings).

References

• Added as part of major Fedora 19 criteria revision
• Test case: missing

Virtualization requirements

Xen DomU

The release must boot successfully as Xen DomU with releases providing a functional, supported Xen Dom0 and widely used cloud providers utilizing Xen.

Dom0 not covered

This criterion specifically does not cover Dom0 functionality: we only absolutely require that DomU functionality must work.

References

• Issue raised 2011-09-26, criterion proposed 2011-10-13, implemented 2011-10-14
• EC2 is by far the most significant use case here
• Test case: QA:Testcase_Boot_Methods_Xen_Para_Virt

Post-install requirements

System services

All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present.

References

• Part of initial Fedora 13 criteria revision
• Rewritten more specifically for major Fedora 19 criteria revision
• Test case: QA:Testcase_Services_start

Keyboard layout configuration

If a particular keyboard layout has been configured for the system, that keyboard layout must be used:

• When unlocking encrypted storage volumes during boot
• When logging in at a console
• When logging in via the default login manager for a release-blocking desktop
• After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)

References

• Proposed 2014-01-14, implemented 2014-02-21
• Test cases: QA:Testcase_base_startup, QA:Testcase_desktop_login

Critical path translations

All critical path actions on release-blocking desktops must correctly display all sufficiently complete translations available for use.

What does this cover, exactly?

This criterion covers bugs that cause available translations not to be shown. It does not require that any translations at all be available: 'something is not translated to my language' cannot constitute a violation of this criterion. The "sufficiently complete" wording refers to a mechanism in Fedora which means that translations are not actually included until they reach a certain percentage of completion.

References

• Proposed 2011-10-12, implemented 2011-10-14
• Test case: missing

SELinux and crash notifications

There must be no SELinux denial notifications or crash notifications on boot of or during installation from a release-blocking live image, or at first login after a default install of a release-blocking desktop.

Exclusions

Notifications that only happen on unusual configurations are excluded: see Blocker_Bug_FAQ.

References

• Part of initial Fedora 13 criteria revision
• Slightly revised for major Fedora 19 criteria revision
• Test case: QA:Testcase_desktop_error_checks

Data corruption

All known bugs that can cause corruption of user data must be fixed or documented at Common F21 bugs.

User data?

"User data" really means data: attempts have been made to apply this criterion to things like trivial items of configuration being reset on upgrades or the setting of the system clock, but that is not going to fly.

Fixed or documented?

If the issue is sufficiently serious, we may consider that documenting it is not sufficient and it must be fixed. This is a subjective determination that will be made at blocker review or Go/No-Go meetings.

References

• Part of initial Fedora 13 criteria revision
• Test case: N/A (a test case for this criterion is not practical, it instead covers all "known bugs")

Default application functionality

All applications that can be launched using the standard graphical mechanism of a release-blocking desktop after a default installation of that desktop must start successfully and withstand a basic functionality test.

All applications installed by default in Fedora Workstation must comply with each MUST and MUST NOT guideline in the Applications and Launchers policy.

Basic functionality

Basic functionality means that the app must at least be broadly capable of its most basic expected operations, and that it must not crash without user intervention or with only basic user intervention.

References

• Part of the 'menu sanity' block of the initial Fedora 13 criteria revision
• Revised to reduce scope as part of major Fedora 19 criteria revision
• Workstation Apps and Launchers policy compliance requirement added 2014-10-01
• Test case: QA:Testcase_desktop_menus

Default panel functionality

All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use.

Typical use?

Especially with GNOME 3 and KDE 4, 'panel (or equivalent)' covers quite a wide range of features, including some pretty advanced stuff - you could argue that all elements of GNOME network configuration are covered because there's a network icon on the top panel, for instance. The intent of the criterion is more that very prominent features of the desktop don't break easily, so there's a subjective cut-off in there which is decided in the blocker review process. The key question is "would this bug cause significant inconvenience or just a really bad first impression to a typical user or reviewer of the release?"

References

• Part of the 'menu sanity' block of the initial Fedora 13 criteria revision
• Test case: QA:Testcase_desktop_panel_advanced

Desktop keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

References

• Part of the 'menu sanity' block of the initial Fedora 13 criteria revision
• Test case: QA:Testcase_desktop_keyring

Artwork

The proposed final Fedora artwork must be included and used as the background on release-blocking desktops. All Fedora artwork visible in critical path actions on release-blocking desktops must be consistent with the proposed final theme.

References

• Proposed 2010-09-10, implemented 2010-09-13
• Test case: QA:Testcase_base_startup

Pre-release notices

No notices or alerts about pre-release status may be shown as part of installation or critical path actions on release-blocking desktops.

Installer warning

This criterion mostly exists to remind us to make sure the 'this is a pre-release, it eats babies' warning in the installer gets taken out before the final release.

References

• Proposed 2011-11-30, implemented 2011-12-15
• Test case: missing

Security bugs

The release must contain no known security bugs of 'important' or higher impact according to the Red Hat severity classification scale which cannot be satisfactorily resolved by a package update (e.g. issues during installation).

Determination

Determination of the classification of a bug can be done by those present at a blocker review or Go/No-Go meeting if necessary, but if the Fedora or Red Hat security team provides a classification, we will usually defer to their wisdom.

References

• Proposed 2012-10-25, implemented 2012-11-08
• Test case: N/A (a test case for this criterion is not practical, it instead covers all "known bugs")

Kickstarts

A spin-kickstarts package which contains the exact kickstart files used to build the release must be present in the release repository. The included kickstarts must define the correct set of release repositories.

Why?

This is considered part of Fedora's duty to be 'self-hosting': the kickstarts used to produce the release images are a vital piece of information required to duplicate that release, so they must be preserved along with the release.

References

• Proposed 2010-10-18, implemented 2011-05-17
• Test case: missing

Release notes

The final branded release notes must be present on release-blocking images and the appropriately versioned generic release notes must be available in the release repository.

References

• Proposed 2010-10-18, implemented 2011-05-17
• Slightly revised as part of major Fedora 19 criteria revision
• Test case: missing

Release identification

A fedora-release package containing the correct names, information and repository configuration for a final Fedora release must be present on release-blocking images and the appropriately versioned generic-release package must be available in the release repository.

References

• Proposed 2010-10-18, implemented 2011-05-17
• Test case: missing

Domain client requirements

These requirements apply to any system properly configured as a client of another system which is an active and correctly configured with the Server Domain Controller role.

Access control

• The system must honor the controller's HBAC rules for access control.

References

• Proposed 2014-11-04, approved 2014-11-18, implemented 2014-11-20
• Test case: QA:Testcase_FreeIPA_realmd_login

Password changes

• Users with domain accounts must be able to change their passwords according to the password policy specified by the domain controller.

References

• Proposed 2014-11-04, approved 2014-11-18, implemented 2014-11-20
• Test case: QA:Testcase_FreeIPA_password_change

Single sign-on

• It must be possible for users to perform passwordless single-sign-on between two properly-configured domain clients using GSSAPI.

References

• Proposed 2014-11-04, approved 2014-11-18, implemented 2014-11-20
• Test case: FIXME

Server discovery

• When configured to use the domain controller for DNS services, the system must be able to use DNS to discover the domain controller address using SRV records.

References

• For this specific requirement, the system need not already be enrolled as a domain client
• Proposed 2014-11-04, approved 2014-11-18, implemented 2014-11-20
• Test case: FIXME

SSH host key validation

• When configured to use FreeIPA for host-key validation, an initial SSH connection between domain clients must not prompt the user to accept the SSH public key.

References

• For this specific requirement, the system need not already be enrolled as a domain client
• Proposed 2014-11-04, approved 2014-11-18, implemented 2014-11-20
• Test case: FIXME

Server Product requirements

These requirements apply only to the Server product.

Domain controller role

Note: role requirements are not expected to live in the Release Criteria in future. The inclusion of requirements for the Server product's initial role is a one-time exception for Fedora 21.

With the Domain Controller role active and correctly configured:

• The Domain Controller must be capable of serving LDAP requests, including TLS-encrypted LDAP requests, on port 389.
• The Domain Controller must be capable of serving LDAPS (LDAP encrypted with SSL) requests on port 636.
• The Domain Controller must be capable of returning LDAP and LDAPS search results using simple auth or SASL/GSSAPI auth.
• The Domain Controller must be capable of serving DNS host records on port 53.
• The Domain Controller must serve DNS SRV records for ldap and kerberos on port 53.

References

• Proposed 2014-11-04, approved 2014-11-18, implemented 2014-11-20
• Test case: FIXME

Final Blocker Bugs

A bug is considered a Final blocker bug if any of the following criteria are met:

• A bug in a Critical Path package that:
  • Cannot be fixed with a future stable update
  • Has a severity rating of high or greater and no reasonable workaround (see definition of severity and priority)
• Bug hinders execution of required Final test plans or dramatically reduces test coverage
• Bug relates to an unmet Final Release Requirement

A Fedora Change being incomplete, in and of itself, does not constitute a blocker bug. The Change process is separate from this process. Changes are required to meet certain standards at certain points of the release cycle, but this is part of the Change process and managed, tracked and enforced separately from this process. However, if a proposed feature being incomplete causes any of the above criteria to be met, then the bug is a release blocker.

Contingency Plan

• If all of the Final Release Requirements are not met by 20:00 UTC on Tuesday the week prior to release day, the release will be delayed by one week so that the Final Release Requirements can be met.
• One week will be added to all remaining tasks in the release schedule, including the final release date.
• This decision will be made at the Go/No-Go Meeting.

Confirming Final Requirements

QA has the responsibility of determining whether the criteria for the release has been met (as outlined above) through discussion with Development and Release Engineering. QA's findings will be reviewed and discussed at the Go/No-Go Meeting.

Related Pages

• Fedora Release Criteria
• Fedora 21 Alpha Release Criteria
• Fedora 21 Beta Release Criteria
• Blocker Bug FAQ
• QA:SOP blocker_bug process
• Automatic blockers
• Packaging:Guidelines