From Fedora Project Wiki

< QA

(remove onboarding call link whose domain went stale)
 
(51 intermediate revisions by 16 users not shown)
Line 1: Line 1:
{{header|qa}}
{{header|qa}}
__NOTOC__


Welcome! This page outlines all the activities you can get involved in to help with [[QA|Fedora QA]]. It's easy to get involved and we'd love to welcome more people to the group, so pick one or more of the activities and jump right in. Please consider joining the [https://admin.fedoraproject.org/mailman/listinfo/test test] and/or the [irc://irc.freenode.net/fedora-qa #fedora-qa IRC channel], so your voice can be heard within the Fedora QA community.
{{autolang|base=yes}}


Whether you are testing a stable release, updates-testing, or Rawhide, you can either use the system as you normally would, or you can choose a component of interest and give it as thorough a testing as you have time to give. Push all the buttons, use all the command line options, verify all the documentation, review it for usability, and suggest future features.  This is especially useful for software which has undergone major changes lately.
<!-- I'm moving a lot of words around here in an attempt to make things easier to find. I want to have simple links for people to click on to get from "Never done QA before" to "I've run my first test or reported my first bug." for whatever component they're working on. Something brought them here, and I want to be able to continue that momentum, letting them get involved with as little friction as possible."


== Reporting bugs in Fedora releases ==
I added a TOC so that we can see what's there easily.


Many people are already involved in Fedora QA, just by reporting problems as you do your regular tasks on Fedora. All you need is a [http://bugzilla.redhat.com Bugzilla] account: [https://bugzilla.redhat.com/createaccount.cgi create your account]. Reporting Fedora bugs as you come across them is a big contribution! We provide some suggestions on [[BugsAndFeatureRequests|reporting bugs]]. If you want to discuss the bugs before reporting them, we can be found on the [https://admin.fedoraproject.org/mailman/listinfo/test test] mailing list and the [irc://irc.freenode.net/fedora-qa #fedora-qa] IRC channel.
--> __NOTOC__


== Joining Test Days ==
= Welcome!=
 
This page outlines all the activities you can get involved in to help with [[QA|Fedora QA]] and is meant to guide you through the QA ecosystem. It's easy to get involved and we love to welcome people to the group! The first thing you probably want to do is get plugged in to all the right sources of information for Fedora QA.
 
There are five steps you can take to get up to speed with QA efforts (in rough order of precedence):
* Create a [[Account_System|FAS account]]
* Subscribe to the {{fplist|test}} mailing list
* [[#self-introduction|Introduce yourself]] to the team!
* Contact to a/several sponsor.
* Create a [https://bugzilla.redhat.com/createaccount.cgi Bugzilla Account] (ideally using the same address as the one associated with your FAS account)
* Join {{matrix|#quality:fedoraproject.org}} Matrix room, see [https://docs.fedoraproject.org/en-US/project/communications/ Communication in Fedora]
* [[#onboarding-call|Attend the onboarding call]]
 
= What are you looking to do? =
 
Whether you are looking to test a [[Releases|stable release]], a new package from [[QA:Updates_Testing|updates-testing]], a [[Releases/Branched|Branched]] [https://fedoraproject.org/get-prerelease pre-release], or the bleeding edge [[Releases/Rawhide|Rawhide]], QA has a way for you to contribute and get involved! We need people who like to push all the buttons, use all the command line options, verify all the documentation, review things for usability, and suggest future features - especially for anything that's undergone major [[Changes/Policy|change]] in a recent or pending release.
 
'''If you want to:'''
* Work on an upcoming stable release
** [[#release-validation|Release Validation Testing]]
** [[QA:Updates_Testing#What_to_test.2C_testing.2C_and_reporting_results|Updates Testing]]
* Work on testing new package updates
** [[QA:Updates_Testing#What_to_test.2C_testing.2C_and_reporting_results|Updates Testing]]
** [[Fedora Easy Karma]]
** [[Fedora Gooey Karma]]
* Work on Rawhide
** All of the above
 
'''I want to do something else!'''
* [[#test-days | Join a Test Day]]
* [[#create-testcases | Create Test Cases]]
* [[#develop-tools | Develop Tools]]
* [[#triage | Triage bugs]]
 
For all of those tasks, it would be good to have some familiarity with our Bugzilla instance. A lot of contribution to Fedora QA comes through bug reports. All you need is a [http://bugzilla.redhat.com Bugzilla] account: [https://bugzilla.redhat.com/createaccount.cgi create your account]. Please make sure that the email addresses for your Fedora account and the Bugzilla account are the same. Our [[BugsAndFeatureRequests| bug reporting practices]] provide some good background for filing bugs. If you want to discuss the bugs before reporting, find QA members in one of our [[QA#Communicate|communication channels]].
 
= Specifics =
 
{{Anchor|self-introduction}}
== Introduce yourself and join the team! ==


The Fedora QA group holds regular Test Days, where we get together on IRC and test a specific aspect of Fedora, often with the involvement of a developer who works in that area. See the [[QA/Test_Days|Test Days]] page for more information on when and where these are held, and how to join in or even schedule one of your own.
Before getting stuck in, why not introduce yourself? It's not compulsory, but we like to say "hi!" to new members. Just subscribe to the {{fplist|test}} mailing list and send a mail with a topic like "Self-introduction: (Your Name)". Just say that you're interested in helping with QA, include your FAS nickname, and, if you like, include some information on your Fedora / Linux background. Here's an [http://www.redhat.com/archives/fedora-test-list/2009-March/msg00671.html example mail].


== NeedsRetesting ==
If you send a self-introduction mail at the same time as you apply to the FAS group, a group sponsor will approve your membership right away. If you do not, a sponsor will contact you directly to confirm your interest before approving your group membership.


Help is needed with for bugs tagged [https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&product=Fedora&version=&component=&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=NEEDINFO&bug_status=MODIFIED&long_desc_type=substring&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwords&status_whiteboard=NeedsRetesting&fixed_in_type=allwordssubstr&fixed_in=&qa_whiteboard_type=allwordssubs NeedsRetesting] in Bugzilla.  These might be from Rawhide or a stable public release. ([http://feeds.feedburner.com/NeedsRetesting NeedsRetesting RSS feed])
{{Anchor|onboarding-call}}
== Attend an Onboarding Call ==


== Testing official updates before they are released ==
Welcoming all newcomers to get a head start and learn the right way to start off with Fedora QA contributions. The onboarding session is one hour of video call with a Fedora QA team member where the new contributors will have an overview of the whole contribution pathway. The idea is to emphasize on the best practices and helping the new contributors figure out the right area of interest. The agenda is very simple and intended to let anyone pick it up without any prerequisites. Watch [https://calendar.fedoraproject.org/QA/ Fedocal] as it will inform about the next onboarding call. Check one of the [https://www.youtube.com/watch?v=ASQmkOrB_DY past recordings].


Another easy way to contribute to Fedora QA is to help test official updates for stable Fedora releases before they're released. See [[QA/Updates Testing]] for instructions on how to test and report issues with these updates.
{{Anchor|updates}}
== Testing official package updates before they are released ==


== Release validation ==
Another easy way to contribute to Fedora QA is to help test official updates for stable Fedora releases before they're released. See [[QA:Updates Testing]] for instructions on how to test and report issues with these updates.


The QA group co-ordinates planned testing of the installation process and basic functionality before each Fedora release and pre-release is made, to ensure they reach certain standards known as the [[Fedora_Release_Criteria]]. See the [[QA:Installation_validation_testing|installation validation testing]] and [[QA:Desktop_validation_testing|desktop validation testing]] pages for more information on these processes and how you can contribute to them.
== Testing Fedora Beta releases ==


== Triaging and managing bugs ==
Before an official Fedora release comes out, a Beta release is made available. You can contribute by installing these Betas and testing them, just as you would a stable release. If you continuously run the pre-release and update regularly you can also [[QA:Update_feedback_guidelines|provide feedback ("karma")]] to [[QA:Updates_Testing|candidate update builds]] - effectively doubling the number of things you can test. When a Beta release is available, you will be able to find it linked from [https://alt.fedoraproject.org/ this page] - if there is no mention of a Beta there, it means there isn't currently one available. Report any issues you find to [http://bugzilla.redhat.com Bugzilla] - complete instructions are here: [[BugsAndFeatureRequests]].


Once bugs are reported, QA makes sure they are addressed by the right people and don't get stuck in the process. The [[BugZappers]] group is responsible for triaging bugs - ensuring they are complete and accurate reports, and assigning them to the right developers. They also shepherd the issues through the process from report to fix released. It's a fun, important job. See the [[BugZappers/Joining|Joining Bugzappers]] page for details on joining in.
{{Anchor|release-validation}}


== Testing Fedora pre-releases ==
== Release validation ==


Before an official Fedora release comes out, several alpha, beta and release candidate releases - known collectively as ''pre-releases'' - are made available. You can contribute by installing these pre-releases and testing them, just as you would a stable release. For information on getting and installing pre-releases, see [http://fedoraproject.org/get-prerelease this page]. Report any issues you find to [http://bugzilla.redhat.com Bugzilla], following the instructions at [[BugsAndFeatureRequests]].
The QA group co-ordinates planned testing of the installation process and basic functionality before each Fedora release and pre-release is made, to ensure they reach certain standards known as the [[Fedora Release Criteria]]. See the [[QA:Release_validation_test_plan|release validation test plan]] page for more information on this process and how you can contribute to it. Also look out for mails to the list that announce composes "nominated for testing" - these mails contain instructions.


{{Anchor|testing-rawhide}}
== Testing Rawhide ==
== Testing Rawhide ==


Rawhide is the development version of Fedora. Running Rawhide isn't for everyone, but for moderately experienced users who have a spare test system available or can run it in a virtual machine, testing Rawhide is a great way to contribute to ensuring future releases will be high quality. See [[Releases/Rawhide]] for instructions on how to install or upgrade to, and test, Rawhide. You can test Rawhide without ever needing to install it by using the [http://alt.fedoraproject.org/pub/alt/nightly-composes/ nightly live builds].
Rawhide is the development version of Fedora. Running Rawhide isn't for everyone, but for moderately experienced users who have a spare test system available or can run it in a virtual machine, testing Rawhide is a great way to contribute to ensuring future releases will be high quality. See [[Releases/Rawhide|Rawhide]] page for instructions on how to install or upgrade to, and test. You can test Rawhide without ever needing to install it by using the [https://apps.fedoraproject.org/releng-dash/ nightly live builds].
 
{{Anchor|test-days}}
== Joining Test Days ==
 
The Fedora QA group holds regular Test Days, where we get together on Matrix and test a specific aspect of Fedora, often with the involvement of a developer who works in that area. See the [[QA/Test_Days|Test Days]] page for more information on when and where these are held, and how to join in or even [[QA/SOP_Test_Day_management|schedule one of your own]]. [[Test_Day:Current|This page]] should always direct you to the current, most recent, or soon-to-happen Test Day - see if it's one you'd like to join in!


{{Anchor|create-testcases}}
== Creating test cases ==
== Creating test cases ==


As well as simply keeping a look out for problems, the QA group develops structured test cases and test plans. See the [[:Category:Test Cases]] and [[:Category:Test Plans]] pages for information on the test cases currently available, and how to get involved with creating new ones.
As well as simply keeping a look out for problems, the QA group develops structured test cases and test plans. See the [[:Category:Test Cases]] and [[:Category:Test Plans]] pages for information on the test cases currently available. The [[QA:SOP_test_case_creation|test case creation process]] will tell you how to create test cases.


{{Anchor|develop-tools}}
== Developing tools ==
== Developing tools ==


Some members of the Fedora QA team are involved in developing and maintaining tools to help make testing more efficient. Some of the tools already developed [https://fedorahosted.org/snake/ SNAKE] and [https://fedorahosted.org/python-bugzilla/ python-bugzilla], and we also use [https://fedorahosted.org/bodhi/ Bodhi] and [https://bugzilla.redhat.com/ Bugzilla]. Tools currently under development include [https://fedorahosted.org/nitrate/ Nitrate], a system for collecting test cases, and [[QA/Beaker|Beaker]], an automated test lab system. Tool development is a great way to apply engineering skills to QA. Contact [[WillWoods|Will]] if you'd like to get involved with building tools for Fedora QA.
Some members of the Fedora QA team are involved in developing and maintaining tools to help make testing more efficient. Some of the tools we wrote that are already in use (but under constant development!) include the [https://pagure.io/fedora-qa/blockerbugs blocker bug tracking web app], a [https://pagure.io/fedora-qa/testdays-web web front end for Test Day results], and [https://pagure.io/fedora-qa/fedora_openqa scheduling tools] for [[OpenQA]]. We also use [https://fedorahosted.org/bodhi/ Bodhi], [https://bugzilla.redhat.com/ Bugzilla] and [[Fedora Easy Karma|fedora-easy-karma]]. See more of them in [[QA:Tools]]. Tool development is a great way to apply engineering skills to QA. [[QA#Communicate|Get in touch with us]] if you want to get involved building tools.
 
{{Anchor|triage}}
 
== Triaging and managing bugs ==
 
Once bugs are reported, QA should make sure they are addressed by the right people and don't get stuck in the process.
 
=== What is Involved in Bug Triaging? ===
 
Bug triagers make sure that:
 
* Bug reports have the information developers need to reproduce and fix them.
* Bugs are assigned to the right component and version.
* Duplicate bugs are found and labelled.
* Feature requests reported as bugs are properly reported.
* Bugs already fixed are closed.
 
The [https://admin.fedoraproject.org/mailman/listinfo/test mailing list] is a great place to raise any questions or problems you are having. Experienced QA Members are more than happy to act as mentors for new members to help them get started with QA. Triaging bugs does '''not''' mean that you have to understand the bugs and solve them yourself. It simply means you should be able to look at new bugs, and report if they are duplicates, if more information is needed, or if it is filed under the wrong component.
 
There is no requirement of programming knowledge. However, being familiar with Fedora and Linux in general will be extremely useful. A more complete guide is [[QA/Triage|here]]. <!-- This could also just point to the mailing list or matrix, but I figured they would just get pointed back to this -->
 
[[Category:QA]]

Latest revision as of 14:16, 14 March 2024


Welcome!

This page outlines all the activities you can get involved in to help with Fedora QA and is meant to guide you through the QA ecosystem. It's easy to get involved and we love to welcome people to the group! The first thing you probably want to do is get plugged in to all the right sources of information for Fedora QA.

There are five steps you can take to get up to speed with QA efforts (in rough order of precedence):

What are you looking to do?

Whether you are looking to test a stable release, a new package from updates-testing, a Branched pre-release, or the bleeding edge Rawhide, QA has a way for you to contribute and get involved! We need people who like to push all the buttons, use all the command line options, verify all the documentation, review things for usability, and suggest future features - especially for anything that's undergone major change in a recent or pending release.

If you want to:

I want to do something else!

For all of those tasks, it would be good to have some familiarity with our Bugzilla instance. A lot of contribution to Fedora QA comes through bug reports. All you need is a Bugzilla account: create your account. Please make sure that the email addresses for your Fedora account and the Bugzilla account are the same. Our bug reporting practices provide some good background for filing bugs. If you want to discuss the bugs before reporting, find QA members in one of our communication channels.

Specifics

Introduce yourself and join the team!

Before getting stuck in, why not introduce yourself? It's not compulsory, but we like to say "hi!" to new members. Just subscribe to the test mailing list and send a mail with a topic like "Self-introduction: (Your Name)". Just say that you're interested in helping with QA, include your FAS nickname, and, if you like, include some information on your Fedora / Linux background. Here's an example mail.

If you send a self-introduction mail at the same time as you apply to the FAS group, a group sponsor will approve your membership right away. If you do not, a sponsor will contact you directly to confirm your interest before approving your group membership.

Attend an Onboarding Call

Welcoming all newcomers to get a head start and learn the right way to start off with Fedora QA contributions. The onboarding session is one hour of video call with a Fedora QA team member where the new contributors will have an overview of the whole contribution pathway. The idea is to emphasize on the best practices and helping the new contributors figure out the right area of interest. The agenda is very simple and intended to let anyone pick it up without any prerequisites. Watch Fedocal as it will inform about the next onboarding call. Check one of the past recordings.

Testing official package updates before they are released

Another easy way to contribute to Fedora QA is to help test official updates for stable Fedora releases before they're released. See QA:Updates Testing for instructions on how to test and report issues with these updates.

Testing Fedora Beta releases

Before an official Fedora release comes out, a Beta release is made available. You can contribute by installing these Betas and testing them, just as you would a stable release. If you continuously run the pre-release and update regularly you can also provide feedback ("karma") to candidate update builds - effectively doubling the number of things you can test. When a Beta release is available, you will be able to find it linked from this page - if there is no mention of a Beta there, it means there isn't currently one available. Report any issues you find to Bugzilla - complete instructions are here: BugsAndFeatureRequests.

Release validation

The QA group co-ordinates planned testing of the installation process and basic functionality before each Fedora release and pre-release is made, to ensure they reach certain standards known as the Fedora Release Criteria. See the release validation test plan page for more information on this process and how you can contribute to it. Also look out for mails to the list that announce composes "nominated for testing" - these mails contain instructions.

Testing Rawhide

Rawhide is the development version of Fedora. Running Rawhide isn't for everyone, but for moderately experienced users who have a spare test system available or can run it in a virtual machine, testing Rawhide is a great way to contribute to ensuring future releases will be high quality. See Rawhide page for instructions on how to install or upgrade to, and test. You can test Rawhide without ever needing to install it by using the nightly live builds.

Joining Test Days

The Fedora QA group holds regular Test Days, where we get together on Matrix and test a specific aspect of Fedora, often with the involvement of a developer who works in that area. See the Test Days page for more information on when and where these are held, and how to join in or even schedule one of your own. This page should always direct you to the current, most recent, or soon-to-happen Test Day - see if it's one you'd like to join in!

Creating test cases

As well as simply keeping a look out for problems, the QA group develops structured test cases and test plans. See the Category:Test Cases and Category:Test Plans pages for information on the test cases currently available. The test case creation process will tell you how to create test cases.

Developing tools

Some members of the Fedora QA team are involved in developing and maintaining tools to help make testing more efficient. Some of the tools we wrote that are already in use (but under constant development!) include the blocker bug tracking web app, a web front end for Test Day results, and scheduling tools for OpenQA. We also use Bodhi, Bugzilla and fedora-easy-karma. See more of them in QA:Tools. Tool development is a great way to apply engineering skills to QA. Get in touch with us if you want to get involved building tools.

Triaging and managing bugs

Once bugs are reported, QA should make sure they are addressed by the right people and don't get stuck in the process.

What is Involved in Bug Triaging?

Bug triagers make sure that:

  • Bug reports have the information developers need to reproduce and fix them.
  • Bugs are assigned to the right component and version.
  • Duplicate bugs are found and labelled.
  • Feature requests reported as bugs are properly reported.
  • Bugs already fixed are closed.

The mailing list is a great place to raise any questions or problems you are having. Experienced QA Members are more than happy to act as mentors for new members to help them get started with QA. Triaging bugs does not mean that you have to understand the bugs and solve them yourself. It simply means you should be able to look at new bugs, and report if they are duplicates, if more information is needed, or if it is filed under the wrong component.

There is no requirement of programming knowledge. However, being familiar with Fedora and Linux in general will be extremely useful. A more complete guide is here.