From Fedora Project Wiki

(→‎Recording Basics: update link to testcase)
(Add Dual Monitor set-up)
 
(2 intermediate revisions by 2 users not shown)
Line 358: Line 358:
* Part of the 'menu sanity' block of the initial Fedora 13 criteria revision
* Part of the 'menu sanity' block of the initial Fedora 13 criteria revision
* Test case: [[QA:Testcase_desktop_panel_advanced]]
* Test case: [[QA:Testcase_desktop_panel_advanced]]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{anchor|dual-monitor-set-up}}
==== Dual Monitor Set-Up ====
On computers using two monitors (e.g. two monitors on a desktop, or the built-in and one external monitor on a laptop), the graphical output is correctly shown on both monitors.
{{hidden|header=References|content=
Due to a wide range of possible hardware configurations, any problem related to this criterion will be regarded as a "conditional blocker" (as described in [https://fedoraproject.org/wiki/Blocker_Bug_FAQ#What_about_hardware_and_local_configuration_dependent_issues.3F What about hardware and local configuration dependent issues.])."
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}


Line 387: Line 394:
* Printing criteria initially proposed [https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/2V6B3DSMPSUFW6IDE32MXQLQ4A4VI7LZ/ 2019-08-20], revised text proposed [https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/SHBBPIIZ6YQO6XROZAWOFQFWSEAPWAYB/ 2019-02-11], implemented [https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/AO3QDTLV7R7JYJK6CPUHOXNFKICE3F5Y/ 2019-04-01]
* Printing criteria initially proposed [https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/2V6B3DSMPSUFW6IDE32MXQLQ4A4VI7LZ/ 2019-08-20], revised text proposed [https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/SHBBPIIZ6YQO6XROZAWOFQFWSEAPWAYB/ 2019-02-11], implemented [https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/AO3QDTLV7R7JYJK6CPUHOXNFKICE3F5Y/ 2019-04-01]
* Test cases: [[QA:Testcase_Printing_New_Printer]] (partially)
* Test cases: [[QA:Testcase_Printing_New_Printer]] (partially)
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{anchor|sound-recording}}
==== Sound recording ====
The installed system must be able to record audio using the default web browser (if applicable) and ''gstreamer-based'' applications.
{{hidden|header=System-specific bugs|content=System-specific bugs don't usually constitute an infringement of this criterion. It is meant to cover bugs which completely prevent sound recording from working in any hardware configuration. See [[Blocker_Bug_FAQ]] for more discussion of this.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{hidden|header=References|content=
* [https://lists.fedorahosted.org/archives/list/test@lists.fedoraproject.org/thread/6X655I5WAR3SDV4YXRYQLAS6EJIOKFW2/ Proposed 2021-01-28], [https://lists.fedorahosted.org/archives/list/test@lists.fedoraproject.org/message/GIGTRQO4JQN4NJJ6PXOGT62TE6TFDGCY/ Implemented 2021-02-23]
* Test case: [https://fedoraproject.org/wiki/QA:Testcase_recording_basic QA:Testcase Recording Basic]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}


Line 441: Line 457:
* Test case: missing
* Test case: missing
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{anchor|recording-basics}}
==== Recording Basics ====
The installed system must be able to record audio using the default web browser (if applicable) and ''gstreamer-based'' applications.
{{hidden|header=References|content=
* [https://lists.fedorahosted.org/archives/list/test@lists.fedoraproject.org/message/2ICU775CHPYW4C766AWCSOYI5NDVKPOR/ Proposed 2021-01-28], [https://fedoraproject.org/w/index.php?title=Fedora_16_Final_Release_Criteria&diff=236660&oldid=236659 implemented 2021-02-23]
* Test case: [https://fedoraproject.org/wiki/QA:Testcase_recording_basic QA:Testcase Recording Basic]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}


{{anchor|user-switching}}
{{anchor|user-switching}}
==== User switching ====
==== User switching ====
User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration.
User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration.

Latest revision as of 10:25, 17 May 2021

Final Objectives

The objective of the Final release is to:

  1. Provide a polished final release suitable for meeting the needs of our Target Audience

Final Release Requirements

In order to be released to the general public, a compose must meet all of the following criteria. This is intended to make the decision process as clear and straightforward as possible. Mostly met items are incomplete until they are met. Optional and nice to have items should not be included in this list.

There may be times where a requirement is unmet only in a particular configuration, such as with some keyboard layouts but not others, or if a particular character is used in a username, password or passphrase. In such cases, the release team should use their judgement and refer to precedent to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by the issue, the severity of the case when the issue is encountered, and the ease or otherwise with which the issue can be avoided by both informed and uninformed users.

The term release-blocking desktops means all the desktop environments in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking desktops for x86_64 is GNOME and KDE, and for aarch64 is GNOME. Note that bugs in desktops that are not part of this set which would infringe these criteria automatically qualify for freeze exception status, according to the freeze exception bug process.

The term release-blocking images means all the images in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking images includes the images defined by the three Editions - Server, Workstation and IoT - in their product requirement documents and/or technical specifications, the Everything network install image, key Cloud images, and the KDE live image. The canonical list of release-blocking images for Fedora 42 is on this page.

Process requirements

Beta criteria met

All Fedora 34 Beta Release Criteria must be met.

Final blockers CLOSED

All bugs blocking the Final tracker must be CLOSED.

Edition requirements

This section contains requirements relating to Fedora Editions: not to any specific edition, but to editions in general.

Self-identification

For each Edition, the installer must be clearly identified as that Edition in all of that Edition's installable release-blocking images. Also, for any deployment of that Edition in accordance with the rest of these criteria, the relevant fields in /etc/os-release must include the correct Edition name. Release-blocking images that are not part of any edition must not identify themselves as being part of any edition.

References

Initialization requirements

Release-blocking images must boot

All release-blocking images must boot in their supported configurations.

Supported media types

Release-blocking live and dedicated installer images must boot when written to optical media of an appropriate size (if applicable) and when written to a USB stick with any of the officially supported methods.

Difference from Beta

This criterion differs from the similar Beta criterion only in that it requires all supported images to work when written also to optical media, not just USB sticks.

QA Testing responsibilities

According to FESCO decision ([1]), QA team is not responsible for physical optical media testing. However, if any bug is found, it'll be considered to be a blocker.

References

Test cases:

'Basic graphics mode' boot mode behavior

The generic video driver option ('basic graphics mode' - as described in the Basic criteria) on all release-blocking installer and live images must function as intended (launching the installer or desktop and attempting to use a generic driver), and there must be no bugs that clearly prevent the installer or desktop from being reached in this configuration on all systems or on wide classes of hardware.

References

No broken packages

There must be no errors in any package on the release-blocking images which cause the package to fail to install.

What errors?

Critical errors include, but are not limited to, undeclared conflicts (explicit Conflicts: tags are acceptable) and unresolved dependencies.

Automatic blockers

Bugs that violate this criterion are usually considered "automatic blockers", they do not have to go through the review process. See QA:SOP_blocker_bug_process#Automatic_blockers for more details on the automatic blocker procedure.

References

Installer requirements

Media consistency verification

Validation of install media must work correctly for all release-blocking images.

Details

This means that the installer's mechanism for verifying that the install medium is intact must complete successfully if the medium is correctly written and return a legible failure message if it is not.

References

Live image persistent overlays

The release-blocking live images must properly support mounting and using a persistent storage overlay for the entire system and/or one for the /home partition.

Writing not covered

This criterion does not cover the writing of the persistent overlay, as that stage is likely to be done using a stable released Fedora or other operating system and hence irrelevant to the release validation process.

References

Package and installer sources

The installer must be able to use all supported local and remote package and installer sources.

References

Network attached storage

The installer must be able to detect (if possible) and install to supported network-attached storage devices.

Supported types

Supported network-attached storage types include iSCSI, Fibre Channel and Fibre Channel over Ethernet (FCoE).

References

Installation interfaces

The installer must be able to complete an installation using all supported interfaces.

References

Minimal install

When installing with the generic network install image with no update repositories enabled, the installer must be able to install the minimal package set.

References

Disk layouts

The installer must be able to create and install to any workable partition layout using any file system and/or container format combination offered in a default installer configuration.

Wait, what?

Yeah, we know. This is a huge catch-all criterion and it's subject to a lot of on-the-fly interpretation. Broadly what it's 'meant to mean' is that you should be able to do anything sane that the Installation Destination spoke attempts to let you do, without the installer exploding or failing. We are trying to write more specific criteria covering this area, but it's not easy. Patches welcome, as the kids say...

Default installer configuration?

The default installer configuration clause specifically excludes options that only appear in the installer if you do something special to trigger them. There have been times when the installer has implemented experimental support for some new filesystem by showing it if you pass a special kernel parameter, for instance: the purpose of this clause is explicitly not to cover cases like that.

References

Firmware RAID

  • The installer must be able to detect and install to firmware RAID storage devices.
System-specific bugs

System-specific bugs don't necessarily constitute an infringement of this criterion. It is not unusual that support for some specific firmware RAID controller, for instance, might be broken. In the case of such system-specific bugs, whether the bug is considered to infringe the criterion will be a subjective decision based on the severity of the bug and how common the hardware in question is considered to be. See Blocker_Bug_FAQ for more discussion of this.

References
  • Partitioning criteria differed prior to Fedora 18 rewrite of installer GUI; hardware, firmware and software RAID were combined
  • Software RAID was rolled into partitioning criteria and firmware/hardware RAID left as a standalone criterion as part of major Fedora 19 criteria revision
  • Firmware RAID requirement move from Beta to Final proposed 2018-09-14, implemented 2018-11-16
  • Test case: QA:Testcase_Install_to_BIOS_RAID

Bootloader disk selection

The installer must allow the user to choose which disk the system bootloader will be installed to, and to choose not to install one at all.

References
  • Added as part of major Fedora 19 criteria revision
  • Test case: missing

Storage volume resize

Any installer mechanism for resizing storage volumes must correctly attempt the requested operation.

Unknown volumes

This criterion specifically does not require the installer to disallow resizing of unformatted volumes or volumes formatted with an unknown filesystem. Nor does it require recognition of any specific filesystems. The installer permitting destructive resizing of such 'unknown' volumes is explicitly not a violation of this criterion. See bug #1033778 and this mailing list thread.

What does that cover?

This means that if the installer offers mechanisms for resizing storage volumes, then it must run the appropriate resizing tool with the appropriate parameters for the resize the user chooses (for volumes with recognized filesystems; see above footnote for behaviour with 'unknown' volumes). The reason it's worded this way is we specifically don't want to cover cases where the requested resize operation then fails for some reason - dirtily unmounted or over-fragmented partition, for instance. We only want to cover the case that the installer's resize code itself is badly broken.

Corruption-causing bugs covered in another criterion

This criterion does not necessarily cover all cases of bugs in resize mechanisms which may cause some kind of data corruption, but any such bug would be covered by the general data corruption criterion.

References

Windows dual boot

The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.

Clean Windows installation?

This is a get-out clause for cases where there's a bug caused by some weirdness or abnormality in the Windows installation; we can't plausibly undertake to support every possible Windows deployment. The expected scenario is a cleanly installed or OEM-deployed Windows installation. Issues caused by recovery or 'system' partitions may not be considered to violate this criterion, depending on the specific circumstances. This criterion is considered to cover both BIOS and UEFI cases.

Secure Boot excepted

The bootloader entry part of this criterion does not apply when Secure Boot is enabled (because it has not yet been made to work, and fixing it is not trivial). cases.

References

OS X dual boot

The installer must be able to install into free space alongside an existing OS X installation, install and configure a bootloader that will boot Fedora.

References

Update image

The installer must be able to use an installer update image retrieved from removable media or a remote package source.

Supplementary to Basic

Note the corresponding Basic criterion that requires update image sourcing via HTTP to work.

References

Installer translations

The installer must correctly display all sufficiently complete translations available for use.

Substantial breakage only

One or two translated strings not being displayed will not usually constitute a violation of this criterion (unless the ones missing are very important strings).

References

Installer help

Any element in the installer interface(s) which is clearly intended to display 'help' text must do so correctly when activated.

References

Cloud image requirements

Cloud images work in EC2

Release-blocking cloud disk images must be published to Amazon EC2 as AMIs, and these must boot successfully and meet other relevant release criteria on at least one KVM-based x86 instance type, at least one KVM-based aarch64 instance type, and at least one Xen-based x86 instance type.

References

Post-install requirements

First boot experience

If an initial setup utility is run or intended to be run after the first boot of the installed system, then it must start successfully and each page or panel of the initial setup utility should withstand a basic functionality test.

References

System services

All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present.

References
  • Part of initial Fedora 13 criteria revision
  • Rewritten more specifically for major Fedora 19 criteria revision
  • Test case: QA:Testcase_Services_start

Keyboard layout configuration

If a particular keyboard layout has been configured for the system, that keyboard layout must be used:

  • When unlocking encrypted storage volumes during boot (but see footnotes)
  • When logging in at a console
  • When logging in via the default login manager for a release-blocking desktop
  • After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)
Switched layouts when unlocked encrypted storage

Note that for technical reasons, 'switched' console layouts - where a special key combination switches between entering one set of characters and another, e.g. Cyrillic and Latin characters - will always be in their default mode when entering the passphrase to unlock an encrypted storage volume, and cannot be switched at that time. This situation does not constitute a violation of the criterion. See bug #681250

References

Critical path translations

All critical path actions on release-blocking desktops must correctly display all sufficiently complete translations available for use.

What does this cover, exactly?

This criterion covers bugs that cause available translations not to be shown. It does not require that any translations at all be available: 'something is not translated to my language' cannot constitute a violation of this criterion. The "sufficiently complete" wording refers to a mechanism in Fedora which means that translations are not actually included until they reach a certain percentage of completion.

References

SELinux and crash notifications

There must be no SELinux denial notifications or crash notifications on boot of or during installation from a release-blocking live image, or at first login after a default install of a release-blocking desktop.

Exclusions

Notifications that only happen on unusual configurations are excluded: see Blocker_Bug_FAQ.

References

Default application functionality

All applications that can be launched using the standard graphical mechanism after a default installation of Fedora Workstation on the x86_64 architecture must start successfully and withstand a basic functionality test.

For other release-blocking desktops (on any architecture), the requirements only apply to the following types of applications:

  • web browser
  • file manager
  • package manager
  • image viewer
  • document viewer
  • text editor
  • archive manager
  • terminal emulator
  • problem reporter
  • help viewer
  • system settings

If there are multiple applications of the same type (e.g. several web browsers), the primary/default one must satisfy the requirements. If the primary/default application can't be determined, at least one of said applications must satisfy the requirements.

All applications installed by default in Fedora Workstation must comply with each MUST and MUST NOT guideline in the Applications and Launchers policy.

Basic functionality

Basic functionality means that the app must at least be broadly capable of its most basic expected operations, and that it must not crash without user intervention or with only basic user intervention.

Application types examples

Here are concrete examples for each application type, so that it is clear what each category contains:

  • web browser: firefox
  • file manager: nautilus
  • package manager: gnome-software
  • image viewer: eog
  • document viewer: evince
  • text editor: gedit
  • archive manager: file-roller
  • terminal emulator: gnome-terminal
  • problem reporter: abrt
  • help viewer: yelp
  • system settings: gnome-control-center
Determining primary applications

The usual way to determine a primary/default application is to look into system configuration where default applications are set (e.g. gnome-control-center -> Default Applications), or to launch the default application on an appropriate file type (e.g. double click on an .odt file in a file manager), or to look into a favorites menu section (e.g. KDE menu -> Favorites).

References

Default panel functionality

All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use.

Typical use?

Especially with GNOME 3 and KDE 4, 'panel (or equivalent)' covers quite a wide range of features, including some pretty advanced stuff - you could argue that all elements of GNOME network configuration are covered because there's a network icon on the top panel, for instance. The intent of the criterion is more that very prominent features of the desktop don't break easily, so there's a subjective cut-off in there which is decided in the blocker review process. The key question is "would this bug cause significant inconvenience or just a really bad first impression to a typical user or reviewer of the release?"

References

Dual Monitor Set-Up

On computers using two monitors (e.g. two monitors on a desktop, or the built-in and one external monitor on a laptop), the graphical output is correctly shown on both monitors.

References

Due to a wide range of possible hardware configurations, any problem related to this criterion will be regarded as a "conditional blocker" (as described in What about hardware and local configuration dependent issues.)."

Update notification

Release-blocking desktops must notify the user of available updates, but must not do so when running as a live image.

References

Desktop keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

References

Printing

Printing must work in release-blocking desktops on at least one printer using each of the following drivers:

  • The built-in print-to-PDF driver
  • The generic IPP driver
'Work'?

'Work' is defined as the printed output from a typical application matching the 'print preview' - note that differences in color reproduction are not considered 'non-working'.

References

Sound recording

The installed system must be able to record audio using the default web browser (if applicable) and gstreamer-based applications.

System-specific bugs

System-specific bugs don't usually constitute an infringement of this criterion. It is meant to cover bugs which completely prevent sound recording from working in any hardware configuration. See Blocker_Bug_FAQ for more discussion of this.

References

Artwork

The proposed final Fedora artwork must be included and used as the background on release-blocking desktops. All Fedora artwork visible in critical path actions on release-blocking desktops must be consistent with the proposed final theme.

References

Pre-release notices

No notices or alerts about pre-release status may be shown as part of installation or critical path actions on release-blocking desktops.

Installer warning

This criterion mostly exists to remind us to make sure the 'this is a pre-release, it eats babies' warning in the installer gets taken out before the final release.

References

Security bugs

The release must contain no known security bugs of 'important' or higher impact according to the Red Hat severity classification scale which cannot be satisfactorily resolved by a package update (e.g. issues during installation).

Determination

Determination of the classification of a bug can be done by those present at a blocker review or Go/No-Go meeting if necessary, but if the Fedora or Red Hat security team provides a classification, we will usually defer to their wisdom.

References

Kickstarts

At the time of the release, the fedora-kickstarts git repository must have a tag matching the commit used to produce the accepted release candidate compose. The tag's name must clearly and unambiguously match the release number.

References
  • Proposed 2010-10-18, implemented 2011-05-17, changed from requiring a package containing the kickstarts to be part of the release to just requiring a git tag 2018-06-27
  • Test case: N/A (not part of the compose, so just releng and QA's responsibility to check it's done by release time)

Release notes

Any release notes included in the release repositories and/or any release-blocking deliverables must be for the correct release, and approved for release by the documentation team.

References

Release identification

A fedora-release package containing the correct names, information and repository configuration for a final Fedora release must be present on release-blocking images and the appropriately versioned generic-release package must be available in the release repository.

References

User switching

User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration.

What is user switching?

User switching is a process of changing the currently presented desktop session between concurrent sessions of two or more different users. The user sessions keep running in the background, and users can switch between them repeatedly without losing any running application state. For the purpose of this criterion, user switching doesn't include switching between different sessions of the same user.

Work?

The switching mechanism must correctly perform the requested operation. If the operation doesn't succeed on a subset of graphical drivers, the release blocking decision should be based on the number of affected users, the problem severity and available workarounds (as is our standard procedure).

Default configuration?

The criterion is supposed to cover only cases where the system hasn't been modified in a substantial (and relevant) way. This excludes cases where people e.g. install multiple desktop environments, replace their display manager for a different one, tweak relevant systemd settings, or install a non-default graphics driver.

References

Domain client requirements

These requirements apply to any system properly configured as a client of another system which is an active and correctly configured FreeIPA domain controller.

Access control

  • The system must honor the controller's HBAC rules for access control.
References

Password changes

  • Users with domain accounts must be able to change their passwords according to the password policy specified by the domain controller.
References

Single sign-on

  • It must be possible for users to perform passwordless single-sign-on between two properly-configured domain clients using GSSAPI.
References

Server discovery

  • When configured to use the domain controller for DNS services, the system must be able to use DNS to discover the domain controller address using SRV records.
References

SSH host key validation

  • When configured to use FreeIPA for host-key validation, an initial SSH connection between domain clients must not prompt the user to accept the SSH public key.
References

Server Edition requirements

These requirements apply only to the Server edition.

FreeIPA and PostgreSQL server requirements

All the additional Beta requirements for FreeIPA and PostgreSQL server systems must be met, without any workarounds being necessary.

References
  • Proposed 2015-01-24, implemented 2015-02-06. Adjusted to avoid reference to Server Roles 2018-07.
  • Test cases: see Beta.

Cockpit management interface

All Cockpit functional criteria must be satisfied when the user is running any of the following platforms:

  • Mozilla Firefox as shipped in the same Fedora release
  • Google Chrome of the latest available version at compose time on the same Fedora release.
  • At least one of a) Mozilla Firefox or b) Google Chrome of the latest available version on Windows at compose time.
  • At least one of a) Mozilla Firefox or b) Google Chrome of the latest available version on OSX at compose time.

Manual testing will occur on the Firefox/Fedora platform. It will not be performed on the Chrome/Fedora platform nor any of the Windows / OSX platforms mentioned above, but if issues in code related to Fedora / Cockpit (and not third-party software such as the third-party browsers) in meeting the functional criteria are reported against those platforms, they may block the release. Workarounds to get around issues that are outside of our control (such as in third party browsers) will not be considered blockers but may be considered freeze exceptions.

References

IoT Edition requirements

Greenboot service

The Greenboot service must be present on all images and installed by default when using the ISO installer. The service must be enabled to run on boot and function as intended.

References

Automatic partition decryption (Clevis)

When configured on hardware with a Trusted Platform Module (TPM), it must be possible for the Clevis utility to automatically decrypt any encrypted partitions on boot without any user intervention.

References


Final Blocker Bugs

A bug is considered a Final blocker bug if any of the following criteria are met:

A Fedora Change being incomplete, in and of itself, does not constitute a blocker bug. The Change process is separate from this process. Changes are required to meet certain standards at certain points of the release cycle, but this is part of the Change process and managed, tracked and enforced separately from this process. However, if a proposed feature being incomplete causes any of the above criteria to be met, then the bug is a release blocker.


Contingency Plan

  • If all of the Final Release Requirements are not met by 20:00 UTC on Tuesday the week prior to release day, the release will be delayed by one week so that the Final Release Requirements can be met.
  • One week will be added to all remaining tasks in the release schedule, including the final release date.
  • This decision will be made at the Go/No-Go Meeting.

Confirming Final Requirements

QA has the responsibility of determining whether the criteria for the release has been met (as outlined above) through discussion with Development and Release Engineering. QA's findings will be reviewed and discussed at the Go/No-Go Meeting.

Related Pages

Retrieved from "https://fedoraproject.org/w/index.php?title=Fedora_34_Final_Release_Criteria&oldid=615415"