From Fedora Project Wiki
(Moved the testing activities under the HOW section) |
(Drop direct Test Days category membership) |
||
| (8 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
== WHAT == | == WHAT == | ||
It's time to kick the tires on encrypted block device installation support in | It's time to kick the tires on encrypted block device installation support in F10-Alpha. There have been several changes on this front recently, most notably the arrival of [[Features/BetterStartup|plymouth]]. The plan is meet and try to poke holes in encrypted block device installations. We'll have a host of QA and Development characters hanging out discussing bugs (aka "features"), expectations, and test areas. I've included details below for how you can contribute below. | ||
F10-Alpha. There have been several changes on this front recently, most | |||
notably the arrival of [[Features/BetterStartup|plymouth]]. The plan is meet | |||
and try to poke holes in encrypted block device installations. We'll have a | |||
host of QA and Development characters hanging out discussing bugs (aka | |||
"features"), expectations, and test areas. I've included details below for | |||
how you can contribute below. | |||
== WHEN == | == WHEN == | ||
| Line 17: | Line 11: | ||
== WHERE == | == WHERE == | ||
Discussion will be held on IRC in the [irc://irc.freenode.net/fedora-qa | Discussion will be held on IRC in the [irc://irc.freenode.net/fedora-qa #fedora-qa] channel. There are quite a few different IRC clients out there you can use to join the discussion, including: | ||
#fedora-qa] channel. There are quite a few different IRC clients out there | |||
you can use to join the discussion, including: | |||
* [http://embed.mibbit.com/?server=irc.freenode.net&channel=%23fedora-qa&forcePrompt=true | * [http://embed.mibbit.com/?server=irc.freenode.net&channel=%23fedora-qa&forcePrompt=true Web-based mibbit] (no additional software required) | ||
* [https://addons.mozilla.org/firefox/16/ Chatzilla firefox extension] | * [https://addons.mozilla.org/firefox/16/ Chatzilla firefox extension] | ||
* A slew of [http://www.ircreviews.org/clients/platforms-unix.html popular | * A slew of [http://www.ircreviews.org/clients/platforms-unix.html popular linux clients] | ||
== WHY == | == WHY == | ||
Have you tried installing using an encrypted block device yet? Support for | Have you tried installing using an encrypted block device yet? Support for [[Anaconda/Features/EncryptedBlockDevices|installing to encrypted block devices]] was added in Fedora 9. While this support has been present for an entire release, it hasn't yet been given a thorough test review. Additionally, with the presence of [[Features/BetterStartup|plymouth]], the method by which unlocking your encrypted devices has changed dramatically. | ||
[[Anaconda/Features/EncryptedBlockDevices|installing to encrypted block | |||
devices]] was added in Fedora 9. While this support has been present for an | |||
entire release, it hasn't yet been given a thorough test review. | |||
Additionally, with the presence of [[Features/BetterStartup|plymouth]], the | |||
method by which unlocking your encrypted devices has changed dramatically. | |||
It's the new hotness and it could use your help in identifying use cases and | It's the new hotness and it could use your help in identifying use cases and fleshing out defects. | ||
fleshing out defects. | |||
== WHO == | == WHO == | ||
The following cast of characters will be available testing, workarounds, bug | The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ... | ||
fixes, and general discussion ... | |||
* Development - dlehman, halfline | * Development - dlehman, halfline | ||
* Quality Assurance - atodorov, bo09, jlaska, mganisin, rwilliam, wwoods | * Quality Assurance - atodorov, bo09, jlaska, mganisin, rwilliam, wwoods | ||
If you'd like to sign up to help field questions, please add your name to the | If you'd like to sign up to help field questions, please add your name to the list above. | ||
list above. | |||
== HOW == | == HOW == | ||
| Line 55: | Line 37: | ||
=== Triage NEW Bugs === | === Triage NEW Bugs === | ||
Much like a field hospital, bug triage is the art of quickly addressing the | Much like a field hospital, bug triage is the art of quickly addressing the big issues in a bug, and moving it on for further review by the development team. Unlike a field hospital, there ''should'' be no blood or guts. The triage process has generated quite a following. You are encouraged to read more about the triage process on the [[BugZappers]] home. In a nutshell, bug triage is a several step process that includes: | ||
big issues in a bug, and moving it on for further review by the development | |||
team. Unlike a field hospital, there ''should'' be no blood or guts. The | |||
triage process has generated quite a following. You are encouraged to read | |||
more about the triage process on the [[BugZappers]] home. In a nutshell, bug | |||
triage is a several step process that includes: | |||
# [[BugZappers/GettingStarted| Getting started]] | # [[BugZappers/GettingStarted| Getting started]] | ||
# [[BugZappers/FindingBugs| Finding bugs to triage]], the list we'll use for | # [[BugZappers/FindingBugs| Finding bugs to triage]], the list we'll use for Test Day includes: | ||
#* [http://tinyurl.com/6k8e5h List of ''NEW'' installer encrypted block device bugs needing triage] | |||
#* [http://tinyurl.com/6k8e5h List of ''NEW'' installer encrypted block device | |||
bugs needing triage] | |||
#* [http://tinyurl.com/6omrxn List of ''NEW'' Plymouth bugs] | #* [http://tinyurl.com/6omrxn List of ''NEW'' Plymouth bugs] | ||
# [[BugZappers/TakingAction| Taking action]] | # [[BugZappers/TakingAction| Taking action]] | ||
One should not spend more than 5 minutes triaging a single bug. If you have | One should not spend more than 5 minutes triaging a single bug. If you have more time to spare, and are looking to get a bit more involved, please proceed with ''Bug Verification'' or ''Test Execution''. | ||
more time to spare, and are looking to get a bit more involved, please proceed | |||
with ''Bug Verification'' or ''Test Execution''. | |||
=== Verify Bug Fixes === | === Verify Bug Fixes === | ||
At this time there are no unverified bug fixes. However, that will likely | At this time there are no unverified bug fixes. However, that will likely change as the days go on. | ||
change as the days go on. | |||
* [http://tinyurl.com/5jfdb7 Encrypted block device installation ''MODIFIED'' | * [http://tinyurl.com/5jfdb7 Encrypted block device installation ''MODIFIED'' bugs] | ||
* [http://tinyurl.com/648k9n Plymouth ''MODIFIED'' bugs] | * [http://tinyurl.com/648k9n Plymouth ''MODIFIED'' bugs] | ||
| Line 87: | Line 58: | ||
==== Getting Started ==== | ==== Getting Started ==== | ||
To help with testing, you'll need to get a copy of the Fedora 10 Alpha. | To help with testing, you'll need to get a copy of the Fedora 10 Alpha. Additionally, since the Test Day involves testing ''installation'', you'll likely need a spare system to install to. | ||
Additionally, since the Test Day involves testing ''installation'', you'll | |||
likely need a spare system to install to. | |||
Here are some testing requirements: | Here are some testing requirements: | ||
* A spare system (can be a virtual guest) | * A spare system (can be a virtual guest) | ||
* Access to install F10-Alpha install media | * Access to install F10-Alpha install media ([http://mirrors.fedoraproject.org//mirrorlists/publiclist//Fedora/10-Alpha mirror list]) | ||
* Optical CD/DVD media and a media burner (optional) | * Optical CD/DVD media and a media burner (optional) | ||
One you have selected a mirror, you must decide how you'd like to start the | One you have selected a mirror, you must decide how you'd like to start the installer. There are many options, but the following lists the most popular choices. | ||
installer. There are many options, but the following lists the most popular | |||
choices. | |||
===== Installation from CD/DVD ===== | ===== Installation from CD/DVD ===== | ||
| Line 108: | Line 73: | ||
* Optical CD/DVD media and a media burner (optional) | * Optical CD/DVD media and a media burner (optional) | ||
The same process used to install an Fedora release can be used to install | The same process used to install an Fedora release can be used to install F10-Alpha. For an overview of the process, please review the [http://docs.fedoraproject.org/install-guide/ Installation Guide]. | ||
F10-Alpha. For an overview of the process, please review the | |||
[http://docs.fedoraproject.org/install-guide/ Installation Guide]. | |||
# First, determine your system architecture | # First, determine your system architecture ([http://docs.fedoraproject.org/install-guide/f9/en_US/sn-which-arch.html docs]). | ||
# Next, locate a nearby mirror where you can download install media ([http://mirrors.fedoraproject.org/publiclist/Fedora/development/ mirror list]). | |||
# Next, [http://docs.fedoraproject.org/install-guide/f9/en_US/sn-expert-download.html download] a ''F10-Alpha'' Fedora CD, DVD, or <code>boot.iso</code> for your architecture from a ([http://docs.fedoraproject.org/install-guide/f9/en_US/sn-expert-download.html docs]) | |||
# Next, locate a nearby mirror where you can download install media | # [http://docs.fedoraproject.org/install-guide/f9/en_US/sn-making-media.html#sn-making-disc-media Burn the downloaded image to a CD/DVD] | ||
# [http://docs.fedoraproject.org/install-guide/f9/en_US/sn-start-installation.html Boot from the CD/DVD] | |||
# Next, | |||
# [http://docs.fedoraproject.org/install-guide/f9/en_US/sn-making-media.html#sn-making-disc-media | |||
# [http://docs.fedoraproject.org/install-guide/f9/en_US/sn-start-installation.html | |||
===== Installation from LiveUSB ===== | ===== Installation from LiveUSB ===== | ||
| Line 136: | Line 88: | ||
This method requires: | This method requires: | ||
* A spare system (can be a virtual guest) - already installed OS (F9, | * A spare system (can be a virtual guest) - already installed OS (F9, F10-Alpha) | ||
Information on installing without | Information on installing without http://docs.fedoraproject.org/install-guide/f9/en_US/ap-medialess-install.html. The same process for locating installation media will be used. | ||
http://docs.fedoraproject.org/install-guide/f9/en_US/ap-medialess-install.html. | |||
The same process for locating installation media will be used. | |||
# First, determine your system architecture (see | # First, determine your system architecture (see http://docs.fedoraproject.org/install-guide/f9/en_US/sn-which-arch.html). | ||
# Next, locate a nearby mirror where you can download install media (see http://mirrors.fedoraproject.org/publiclist/Fedora/development/). | |||
# Next, locate a nearby mirror where you can download install media (see | |||
# Next, install the <code>snake</code> utility: <pre># yum install snake</pre> | # Next, install the <code>snake</code> utility: <pre># yum install snake</pre> | ||
# Using the mirror you selected above, update your bootloader information | # Using the mirror you selected above, update your bootloader information using snake: <pre># snake-install http://download.fedora.redhat.com/pub/fedora/linux/releases/test/10-Alpha/Fedora/i386/os</pre> | ||
# Reboot your system to start the install | # Reboot your system to start the install | ||
| Line 157: | Line 102: | ||
==== Exploratory Testing ==== | ==== Exploratory Testing ==== | ||
Exploratory testing is an approach to software testing that is not scripted or | Exploratory testing is an approach to software testing that is not scripted or planned in as much detail as a more traditional test plan. Rather than walking through a series of pre-defined test cases, the tester is asked to think about the high-level test areas in the software. From there, the tester is encouraged to use their knowledge of the product and the code in order to navigate through different areas of the software. As they walk through the software, their emphasis is to find bugs, expose new test areas, and learn more about the product to help guide future test efforts. | ||
planned in as much detail as a more traditional test plan. Rather than | |||
walking through a series of pre-defined test cases, the tester is asked to | |||
think about the high-level test areas in the software. From there, the tester | |||
is encouraged to use their knowledge of the product and the code in order to | |||
navigate through different areas of the software. As they walk through the | |||
software, their emphasis is to find bugs, expose new test areas, and learn | |||
more about the product to help guide future test efforts. | |||
The simplest definition from [http://www.satisfice.com/articles/et-article.pdf | The simplest definition from [http://www.satisfice.com/articles/et-article.pdf Exploratory Testing Explained by James Bach] works the best: | ||
Exploratory Testing Explained by James Bach] works the best: | |||
''Exploratory testing is simultaneous learning, test design, and test | ''Exploratory testing is simultaneous learning, test design, and test execution.'' | ||
execution.'' | |||
For further reading on exploratory testing can be found at | For further reading on exploratory testing can be found at http://en.wikipedia.org/wiki/Exploratory_testing. | ||
http://en.wikipedia.org/wiki/Exploratory_testing. | |||
===== Test Areas ===== | ===== Test Areas ===== | ||
For our test day, let's define the following test areas as primary focus areas | For our test day, let's define the following test areas as primary focus areas for exploratory testing: | ||
for exploratory testing: | |||
# '''Software | # '''Software [http://en.wikipedia.org/wiki/Redundant_array_of_independent_disks RAID]''' - mix encrypted raid devices, raid members | ||
# '''[http://en.wikipedia.org/wiki/Logical_volume_management LVM]''' - physical volumes, logical volumes | |||
# '''File system probing''' - Scanning a filesystem that may, or may not, contain encrypted block devices (includes rescue-mode) | |||
# '''[http://en.wikipedia.org/wiki/Logical_volume_management LVM]''' - | |||
# '''File system probing''' - Scanning a filesystem that may, or may not, | |||
# '''Rescue mode''' - can we find and use encrypted devices? | # '''Rescue mode''' - can we find and use encrypted devices? | ||
# '''Passphrase prompt''' - anything related to passphrase prompting, | # '''Passphrase prompt''' - anything related to passphrase prompting, different keyboards, languages, hitting <back> or <cancel> several times | ||
# '''SELinux''' - different policies, permissive, enforcing ... does this affect storing or checking the passphrase? | |||
# '''SELinux''' - different policies, permissive, enforcing ... does this | |||
==== Targeted Testing ==== | ==== Targeted Testing ==== | ||
Not in the mood for exploratory testing? That's fine. We've taken the time | Not in the mood for exploratory testing? That's fine. We've taken the time to define explicit test cases for the different test areas noted above. | ||
to define explicit test cases for the different test areas noted above. | |||
With a spare system(s) in hand, you are welcome to walk through one or more of | With a spare system(s) in hand, you are welcome to walk through one or more of the following test cases. | ||
the following test cases. | |||
* [[Anaconda/Features/EncryptedBlockDevices#Test Plan|Encrypted block device | * [[Anaconda/Features/EncryptedBlockDevices#Test Plan|Encrypted block device installation tests cases]] | ||
* [[Features/BetterStartup#Test_Plan|Plymouth test cases]] | * [[Features/BetterStartup#Test_Plan|Plymouth test cases]] | ||
The goal is to flesh out any bugs in the software that might be exposed by | The goal is to flesh out any bugs in the software that might be exposed by your special hardware environment. | ||
your special hardware environment. | |||
==== Test review ==== | ==== Test review ==== | ||
See a typo in the proposed test matrix? Have an interesting hardware scenario | See a typo in the proposed test matrix? Have an interesting hardware scenario to share? Is something not clear? Help us improve the test coverage. | ||
to share? Is something not clear? Help us improve the test coverage. | |||
Come join us on #fedora-qa, ask questions, propose changes, and share your | Come join us on #fedora-qa, ask questions, propose changes, and share your issues. | ||
issues. | |||
=== Known Issues === | === Known Issues === | ||
Both encrypted block device installation, and plymouth have been available for | Both encrypted block device installation, and plymouth have been available for testing already. As a result, there is a list of bugs already filed for each area. Before you file a new bug, please refer to the following before filing a new bug: | ||
testing already. As a result, there is a list of bugs already filed for each | |||
area. Before you file a new bug, please refer to the following before filing | |||
a new bug: | |||
* [http://tinyurl.com/5euz7s Plymouth known issues] | * [http://tinyurl.com/5euz7s Plymouth known issues] | ||
* [http://tinyurl.com/5wwt3y Installer encrypted block devices known issues] | * [http://tinyurl.com/5wwt3y Installer encrypted block devices known issues] | ||
[[Category:Fedora 10 Test Days]] | |||
Latest revision as of 19:40, 26 June 2015
WHAT[edit]
It's time to kick the tires on encrypted block device installation support in F10-Alpha. There have been several changes on this front recently, most notably the arrival of plymouth. The plan is meet and try to poke holes in encrypted block device installations. We'll have a host of QA and Development characters hanging out discussing bugs (aka "features"), expectations, and test areas. I've included details below for how you can contribute below.
WHEN[edit]
DATE: Thursday, August 14th, 2008
TIME: Between 12:00 and 21:00 UTC (8am -> 5pm EDT)
WHERE[edit]
Discussion will be held on IRC in the #fedora-qa channel. There are quite a few different IRC clients out there you can use to join the discussion, including:
- Web-based mibbit (no additional software required)
- Chatzilla firefox extension
- A slew of popular linux clients
WHY[edit]
Have you tried installing using an encrypted block device yet? Support for installing to encrypted block devices was added in Fedora 9. While this support has been present for an entire release, it hasn't yet been given a thorough test review. Additionally, with the presence of plymouth, the method by which unlocking your encrypted devices has changed dramatically.
It's the new hotness and it could use your help in identifying use cases and fleshing out defects.
WHO[edit]
The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ...
- Development - dlehman, halfline
- Quality Assurance - atodorov, bo09, jlaska, mganisin, rwilliam, wwoods
If you'd like to sign up to help field questions, please add your name to the list above.
HOW[edit]
There are many ways you can help, depending on your interests and skill level.
Triage NEW Bugs[edit]
Much like a field hospital, bug triage is the art of quickly addressing the big issues in a bug, and moving it on for further review by the development team. Unlike a field hospital, there should be no blood or guts. The triage process has generated quite a following. You are encouraged to read more about the triage process on the BugZappers home. In a nutshell, bug triage is a several step process that includes:
- Getting started
- Finding bugs to triage, the list we'll use for Test Day includes:
- Taking action
One should not spend more than 5 minutes triaging a single bug. If you have more time to spare, and are looking to get a bit more involved, please proceed with Bug Verification or Test Execution.
Verify Bug Fixes[edit]
At this time there are no unverified bug fixes. However, that will likely change as the days go on.
Testing[edit]
Getting Started[edit]
To help with testing, you'll need to get a copy of the Fedora 10 Alpha. Additionally, since the Test Day involves testing installation, you'll likely need a spare system to install to.
Here are some testing requirements:
- A spare system (can be a virtual guest)
- Access to install F10-Alpha install media (mirror list)
- Optical CD/DVD media and a media burner (optional)
One you have selected a mirror, you must decide how you'd like to start the installer. There are many options, but the following lists the most popular choices.
Installation from CD/DVD[edit]
This method requires:
- A spare system (can be a virtual guest)
- Optical CD/DVD media and a media burner (optional)
The same process used to install an Fedora release can be used to install F10-Alpha. For an overview of the process, please review the Installation Guide.
- First, determine your system architecture (docs).
- Next, locate a nearby mirror where you can download install media (mirror list).
- Next, download a F10-Alpha Fedora CD, DVD, or
boot.isofor your architecture from a (docs) - Burn the downloaded image to a CD/DVD
- Boot from the CD/DVD
Installation from LiveUSB[edit]
- Refer to USB HowTo
Installation over Network[edit]
This method requires:
- A spare system (can be a virtual guest) - already installed OS (F9, F10-Alpha)
Information on installing without http://docs.fedoraproject.org/install-guide/f9/en_US/ap-medialess-install.html. The same process for locating installation media will be used.
- First, determine your system architecture (see http://docs.fedoraproject.org/install-guide/f9/en_US/sn-which-arch.html).
- Next, locate a nearby mirror where you can download install media (see http://mirrors.fedoraproject.org/publiclist/Fedora/development/).
- Next, install the
snakeutility:# yum install snake
- Using the mirror you selected above, update your bootloader information using snake:
# snake-install http://download.fedora.redhat.com/pub/fedora/linux/releases/test/10-Alpha/Fedora/i386/os
- Reboot your system to start the install
With your system installed, you are ready to move on to testing. Good luck!
Exploratory Testing[edit]
Exploratory testing is an approach to software testing that is not scripted or planned in as much detail as a more traditional test plan. Rather than walking through a series of pre-defined test cases, the tester is asked to think about the high-level test areas in the software. From there, the tester is encouraged to use their knowledge of the product and the code in order to navigate through different areas of the software. As they walk through the software, their emphasis is to find bugs, expose new test areas, and learn more about the product to help guide future test efforts.
The simplest definition from Exploratory Testing Explained by James Bach works the best:
Exploratory testing is simultaneous learning, test design, and test execution.
For further reading on exploratory testing can be found at http://en.wikipedia.org/wiki/Exploratory_testing.
Test Areas[edit]
For our test day, let's define the following test areas as primary focus areas for exploratory testing:
- Software RAID - mix encrypted raid devices, raid members
- LVM - physical volumes, logical volumes
- File system probing - Scanning a filesystem that may, or may not, contain encrypted block devices (includes rescue-mode)
- Rescue mode - can we find and use encrypted devices?
- Passphrase prompt - anything related to passphrase prompting, different keyboards, languages, hitting <back> or <cancel> several times
- SELinux - different policies, permissive, enforcing ... does this affect storing or checking the passphrase?
Targeted Testing[edit]
Not in the mood for exploratory testing? That's fine. We've taken the time to define explicit test cases for the different test areas noted above.
With a spare system(s) in hand, you are welcome to walk through one or more of the following test cases.
The goal is to flesh out any bugs in the software that might be exposed by your special hardware environment.
Test review[edit]
See a typo in the proposed test matrix? Have an interesting hardware scenario to share? Is something not clear? Help us improve the test coverage.
Come join us on #fedora-qa, ask questions, propose changes, and share your issues.
Known Issues[edit]
Both encrypted block device installation, and plymouth have been available for testing already. As a result, there is a list of bugs already filed for each area. Before you file a new bug, please refer to the following before filing a new bug: