From Fedora Project Wiki
(Import F27 release criteria.)
 
(Add modularity section)
Line 437: Line 437:
   
   
Manual testing will occur on the Firefox/Fedora platform. It will not be performed on the Chrome/Fedora platform nor any of the Windows / OSX platforms mentioned above, but if issues in code related to Fedora / Cockpit (and not third-party software such as the third-party browsers) in meeting the functional criteria are reported against those platforms, they may block the release. Workarounds to get around issues that are outside of our control (such as in third party browsers) will not be considered blockers but may be considered freeze exceptions.
Manual testing will occur on the Firefox/Fedora platform. It will not be performed on the Chrome/Fedora platform nor any of the Windows / OSX platforms mentioned above, but if issues in code related to Fedora / Cockpit (and not third-party software such as the third-party browsers) in meeting the functional criteria are reported against those platforms, they may block the release. Workarounds to get around issues that are outside of our control (such as in third party browsers) will not be considered blockers but may be considered freeze exceptions.
{{anchor|modularity-product-requirements}}
=== <span style="text-decoration:underline">[[Modularity]] Product requirements</span> ===
As modularity is still based on normal packaging most of the modular distribution testing should be performed using the instructions above, but note that given the packages for the modular distribution are different from the non-modular distributions this means testing twice while we are releasing two distributions.
{{anchor|repository-modular-data}}
==== Repositories should have modular data ====
The installer and packaging infrastructure must be able to use all supported local and remote modularity data, mainly from installed repositories.
* Getting a list of modules.
* Installing a module (and have the defaults work correctly, if the entire modular specification isn't provided).
* Upgrading a module.
* Removing a module.
* Switching the stream of a module (if a module is provided in more than a single stream for a release).


{{hidden|header=References|content=
{{hidden|header=References|content=

Revision as of 03:47, 18 September 2017

Final Objectives

The objective of the Final release is to:

  1. Provide a polished final release suitable for meeting the needs of our Target Audience

Final Release Requirements

In order to be released to the general public, a compose must meet all of the following criteria. This is intended to make the decision process as clear and straightforward as possible. Mostly met items are incomplete until they are met. Optional and nice to have items should not be included in this list.

There may be times where a requirement is unmet only in a particular configuration, such as with some keyboard layouts but not others, or if a particular character is used in a username, password or passphrase. In such cases, the release team should use their judgement and refer to precedent to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by the issue, the severity of the case when the issue is encountered, and the ease or otherwise with which the issue can be avoided by both informed and uninformed users.

The term release-blocking desktops means all the desktop environments in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking desktops for x86_64 is GNOME and KDE, and for aarch64 is GNOME. Note that bugs in desktops that are not part of this set which would infringe these criteria automatically qualify for freeze exception status, according to the freeze exception bug process.

The term release-blocking images means all the images in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking images includes the images defined by the three Editions - Server, Workstation and IoT - in their product requirement documents and/or technical specifications, the Everything network install image, key Cloud images, and the KDE live image. The canonical list of release-blocking images for Fedora 42 is on this page.

Process requirements

Beta criteria met

All Fedora 27 Beta Release Criteria must be met.

Final blockers CLOSED

All bugs blocking the Final tracker must be CLOSED.

Initialization requirements

Release-blocking images must boot

All release-blocking images must boot in their supported configurations.

Supported media types

Release-blocking live and dedicated installer images must boot when written to optical media of an appropriate size (if applicable) and when written to a USB stick with any of the officially supported methods.

Difference from Beta

This criterion differs from the similar Beta criterion only in that it requires all supported images to work when written also to optical media, not just USB sticks.

References

Test cases:

No broken packages

There must be no errors in any package on the release-blocking images which cause the package to fail to install.

What errors?

Critical errors include, but are not limited to, undeclared conflicts (explicit Conflicts: tags are acceptable) and unresolved dependencies.

Automatic blockers

Bugs that violate this criterion are usually considered "automatic blockers", they do not have to go through the review process. See QA:SOP_blocker_bug_process#Automatic_blockers for more details on the automatic blocker procedure.

References

Installer requirements

Media consistency verification

Validation of install media must work correctly for all release-blocking images.

Details

This means that the installer's mechanism for verifying that the install medium is intact must complete successfully if the medium is correctly written and return a legible failure message if it is not.

References

Live image persistent overlays

The release-blocking live images must properly support mounting and using a persistent storage overlay for the entire system and/or one for the /home partition.

Writing not covered

This criterion does not cover the writing of the persistent overlay, as that stage is likely to be done using a stable released Fedora or other operating system and hence irrelevant to the release validation process.

References

Package and installer sources

The installer must be able to use all supported local and remote package and installer sources.

References

Network attached storage

The installer must be able to detect (if possible) and install to supported network-attached storage devices.

Supported types

Supported network-attached storage types include iSCSI, Fibre Channel and Fibre Channel over Ethernet (FCoE).

References

Installation interfaces

The installer must be able to complete an installation using all supported interfaces.

References

Minimal install

When installing with the generic network install image with no update repositories enabled, the installer must be able to install the minimal package set.

References

Disk layouts

The installer must be able to create and install to any workable partition layout using any file system and/or container format combination offered in a default installer configuration.

Wait, what?

Yeah, we know. This is a huge catch-all criterion and it's subject to a lot of on-the-fly interpretation. Broadly what it's 'meant to mean' is that you should be able to do anything sane that the Installation Destination spoke attempts to let you do, without the installer exploding or failing. We are trying to write more specific criteria covering this area, but it's not easy. Patches welcome, as the kids say...

Default installer configuration?

The default installer configuration clause specifically excludes options that only appear in the installer if you do something special to trigger them. There have been times when the installer has implemented experimental support for some new filesystem by showing it if you pass a special kernel parameter, for instance: the purpose of this clause is explicitly not to cover cases like that.

References

Bootloader disk selection

The installer must allow the user to choose which disk the system bootloader will be installed to, and to choose not to install one at all.

References
  • Added as part of major Fedora 19 criteria revision
  • Test case: missing

Storage volume resize

Any installer mechanism for resizing storage volumes must correctly attempt the requested operation.

Unknown volumes

This criterion specifically does not require the installer to disallow resizing of unformatted volumes or volumes formatted with an unknown filesystem. Nor does it require recognition of any specific filesystems. The installer permitting destructive resizing of such 'unknown' volumes is explicitly not a violation of this criterion. See bug #1033778 and this mailing list thread.

What does that cover?

This means that if the installer offers mechanisms for resizing storage volumes, then it must run the appropriate resizing tool with the appropriate parameters for the resize the user chooses (for volumes with recognized filesystems; see above footnote for behaviour with 'unknown' volumes). The reason it's worded this way is we specifically don't want to cover cases where the requested resize operation then fails for some reason - dirtily unmounted or over-fragmented partition, for instance. We only want to cover the case that the installer's resize code itself is badly broken.

Corruption-causing bugs covered in another criterion

This criterion does not necessarily cover all cases of bugs in resize mechanisms which may cause some kind of data corruption, but any such bug would be covered by the general data corruption criterion.

References

Windows dual boot

The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.

Clean Windows installation?

This is a get-out clause for cases where there's a bug caused by some weirdness or abnormality in the Windows installation; we can't plausibly undertake to support every possible Windows deployment. The expected scenario is a cleanly installed or OEM-deployed Windows installation. Issues caused by recovery or 'system' partitions may not be considered to violate this criterion, depending on the specific circumstances. This criterion is considered to cover both BIOS and UEFI cases.

Secure Boot excepted

The bootloader entry part of this criterion does not apply when Secure Boot is enabled (because it has not yet been made to work, and fixing it is not trivial). cases.

References

OS X dual boot

The installer must be able to install into free space alongside an existing OS X installation, install and configure a bootloader that will boot Fedora.

References

Update image

The installer must be able to use an installer update image retrieved from removable media or a remote package source.

Supplementary to Alpha

Note the corresponding Alpha criterion that requires update image sourcing via HTTP to work.

References

Installer translations

The installer must correctly display all sufficiently complete translations available for use.

Substantial breakage only

One or two translated strings not being displayed will not usually constitute a violation of this criterion (unless the ones missing are very important strings).

References

Installer help

Any element in the installer interface(s) which is clearly intended to display 'help' text must do so correctly when activated.

References

Virtualization requirements

Xen DomU

The release must boot successfully as Xen DomU with releases providing a functional, supported Xen Dom0 and widely used cloud providers utilizing Xen.

Dom0 not covered

This criterion specifically does not cover Dom0 functionality: we only absolutely require that DomU functionality must work.

References

Post-install requirements

System services

All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present.

References
  • Part of initial Fedora 13 criteria revision
  • Rewritten more specifically for major Fedora 19 criteria revision
  • Test case: QA:Testcase_Services_start

Keyboard layout configuration

If a particular keyboard layout has been configured for the system, that keyboard layout must be used:

  • When unlocking encrypted storage volumes during boot (but see footnotes)
  • When logging in at a console
  • When logging in via the default login manager for a release-blocking desktop
  • After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)
Switched layouts when unlocked encrypted storage

Note that for technical reasons, 'switched' console layouts - where a special key combination switches between entering one set of characters and another, e.g. Cyrillic and Latin characters - will always be in their default mode when entering the passphrase to unlock an encrypted storage volume, and cannot be switched at that time. This situation does not constitute a violation of the criterion. See bug #681250

References

Critical path translations

All critical path actions on release-blocking desktops must correctly display all sufficiently complete translations available for use.

What does this cover, exactly?

This criterion covers bugs that cause available translations not to be shown. It does not require that any translations at all be available: 'something is not translated to my language' cannot constitute a violation of this criterion. The "sufficiently complete" wording refers to a mechanism in Fedora which means that translations are not actually included until they reach a certain percentage of completion.

References

SELinux and crash notifications

There must be no SELinux denial notifications or crash notifications on boot of or during installation from a release-blocking live image, or at first login after a default install of a release-blocking desktop.

Exclusions

Notifications that only happen on unusual configurations are excluded: see Blocker_Bug_FAQ.

References

Data corruption

All known bugs that can cause corruption of user data must be fixed or documented at Common F27 bugs.

User data?

"User data" really means data: attempts have been made to apply this criterion to things like trivial items of configuration being reset on upgrades or the setting of the system clock, but that is not going to fly.

Fixed or documented?

If the issue is sufficiently serious, we may consider that documenting it is not sufficient and it must be fixed. This is a subjective determination that will be made at blocker review or Go/No-Go meetings.

References
  • Part of initial Fedora 13 criteria revision
  • Test case: N/A (a test case for this criterion is not practical, it instead covers all "known bugs")

Default application functionality

All applications that can be launched using the standard graphical mechanism of a release-blocking desktop after a default installation of that desktop must start successfully and withstand a basic functionality test.

All applications installed by default in Fedora Workstation must comply with each MUST and MUST NOT guideline in the Applications and Launchers policy.

Basic functionality

Basic functionality means that the app must at least be broadly capable of its most basic expected operations, and that it must not crash without user intervention or with only basic user intervention.

References
  • Part of the 'menu sanity' block of the initial Fedora 13 criteria revision
  • Revised to reduce scope as part of major Fedora 19 criteria revision
  • Workstation Apps and Launchers policy compliance requirement added 2014-10-01
  • Test case: QA:Testcase_desktop_menus

Default panel functionality

All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use.

Typical use?

Especially with GNOME 3 and KDE 4, 'panel (or equivalent)' covers quite a wide range of features, including some pretty advanced stuff - you could argue that all elements of GNOME network configuration are covered because there's a network icon on the top panel, for instance. The intent of the criterion is more that very prominent features of the desktop don't break easily, so there's a subjective cut-off in there which is decided in the blocker review process. The key question is "would this bug cause significant inconvenience or just a really bad first impression to a typical user or reviewer of the release?"

References

Update notification

Release-blocking desktops must notify the user of available updates, but must not do so when running as a live image.

References

Desktop keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

References

Artwork

The proposed final Fedora artwork must be included and used as the background on release-blocking desktops. All Fedora artwork visible in critical path actions on release-blocking desktops must be consistent with the proposed final theme.

References

Pre-release notices

No notices or alerts about pre-release status may be shown as part of installation or critical path actions on release-blocking desktops.

Installer warning

This criterion mostly exists to remind us to make sure the 'this is a pre-release, it eats babies' warning in the installer gets taken out before the final release.

References

Security bugs

The release must contain no known security bugs of 'important' or higher impact according to the Red Hat severity classification scale which cannot be satisfactorily resolved by a package update (e.g. issues during installation).

Determination

Determination of the classification of a bug can be done by those present at a blocker review or Go/No-Go meeting if necessary, but if the Fedora or Red Hat security team provides a classification, we will usually defer to their wisdom.

References

Kickstarts

A spin-kickstarts package which contains the exact kickstart files used to build the release must be present in the release repository. The included kickstarts must define the correct set of release repositories.

Why?

This is considered part of Fedora's duty to be 'self-hosting': the kickstarts used to produce the release images are a vital piece of information required to duplicate that release, so they must be preserved along with the release.

References

Release notes

The final branded and generic release notes must be present in the release repository.

References

Release identification

A fedora-release package containing the correct names, information and repository configuration for a final Fedora release must be present on release-blocking images and the appropriately versioned generic-release package must be available in the release repository.

References

Domain client requirements

These requirements apply to any system properly configured as a client of another system which is an active and correctly configured with the Server Domain Controller role.

Access control

  • The system must honor the controller's HBAC rules for access control.
References

Password changes

  • Users with domain accounts must be able to change their passwords according to the password policy specified by the domain controller.
References

Single sign-on

  • It must be possible for users to perform passwordless single-sign-on between two properly-configured domain clients using GSSAPI.
References

Server discovery

  • When configured to use the domain controller for DNS services, the system must be able to use DNS to discover the domain controller address using SRV records.
References

SSH host key validation

  • When configured to use FreeIPA for host-key validation, an initial SSH connection between domain clients must not prompt the user to accept the SSH public key.
References

Server Product requirements

These requirements apply only to the Server product.

Role functional requirements

All functional requirements for all Featured Server Roles must be met, without any workarounds being necessary.

References
  • Proposed 2015-01-24, implemented 2015-02-06.
  • Test cases: see each role's requirements page.

Cockpit management interface

All Cockpit functional criteria must be satisfied when the user is running any of the following platforms:

  • Mozilla Firefox as shipped in the same Fedora release
  • Google Chrome of the latest available version at compose time on the same Fedora release.
  • At least one of a) Mozilla Firefox or b) Google Chrome of the latest available version on Windows at compose time.
  • At least one of a) Mozilla Firefox or b) Google Chrome of the latest available version on OSX at compose time.

Manual testing will occur on the Firefox/Fedora platform. It will not be performed on the Chrome/Fedora platform nor any of the Windows / OSX platforms mentioned above, but if issues in code related to Fedora / Cockpit (and not third-party software such as the third-party browsers) in meeting the functional criteria are reported against those platforms, they may block the release. Workarounds to get around issues that are outside of our control (such as in third party browsers) will not be considered blockers but may be considered freeze exceptions.


Modularity Product requirements

As modularity is still based on normal packaging most of the modular distribution testing should be performed using the instructions above, but note that given the packages for the modular distribution are different from the non-modular distributions this means testing twice while we are releasing two distributions.

Repositories should have modular data

The installer and packaging infrastructure must be able to use all supported local and remote modularity data, mainly from installed repositories.

  • Getting a list of modules.
  • Installing a module (and have the defaults work correctly, if the entire modular specification isn't provided).
  • Upgrading a module.
  • Removing a module.
  • Switching the stream of a module (if a module is provided in more than a single stream for a release).
References


Final Blocker Bugs

A bug is considered a Final blocker bug if any of the following criteria are met:

A Fedora Change being incomplete, in and of itself, does not constitute a blocker bug. The Change process is separate from this process. Changes are required to meet certain standards at certain points of the release cycle, but this is part of the Change process and managed, tracked and enforced separately from this process. However, if a proposed feature being incomplete causes any of the above criteria to be met, then the bug is a release blocker.


Contingency Plan

  • If all of the Final Release Requirements are not met by 20:00 UTC on Tuesday the week prior to release day, the release will be delayed by one week so that the Final Release Requirements can be met.
  • One week will be added to all remaining tasks in the release schedule, including the final release date.
  • This decision will be made at the Go/No-Go Meeting.

Confirming Final Requirements

QA has the responsibility of determining whether the criteria for the release has been met (as outlined above) through discussion with Development and Release Engineering. QA's findings will be reviewed and discussed at the Go/No-Go Meeting.

Related Pages

Retrieved from "https://fedoraproject.org/w/index.php?title=User:James/Draft:Fedora_27_Final_Release_Criteria%2Bmodularity&oldid=501504"