From Fedora Project Wiki

< QA‎ | TestPlans

m (Added performance test area)
 
(40 intermediate revisions by 11 users not shown)
Line 1: Line 1:
= Introduction =
= Introduction =


This document describes the test process by which Fedora 10 will be verified for operation consistent to that of other common laptop hardware.
This document describes the test process by which Fedora 10 will be verified for operation consistent to that of other common laptop hardware. (For information regarding Fedora 11 and beyond on XO hardware, see [[OLPC/Fedora_on_XO]].)


The goals of this plan are to:
The goals of this plan are to:
Line 50: Line 50:
* Installation of Fedora on an XO - only live image booting from the SD is planned for Fedora 10
* Installation of Fedora on an XO - only live image booting from the SD is planned for Fedora 10


{{admon/important|FIXME|Do we need to test persistent storage on the live image?}}
= Responsibilities =


= Test Areas =
== Test Teams ==


== [[QA/TestPlans/Camera | Built-in Camera]] ==
This section outlines the test teams and various community points of contact.  If you are an XO Tester, please add your name to one of the groups on the [[XO Test Roll Call]].


* What applications can be used to verify the camera?
== Team Lead ==
** Possibly Cheese. http://www.gnome.org/projects/cheese/
* Laptop microphone/camera in-use indicators behave as expected


== [[QA/TestPlans/Power Management | Power Management]] ==
Responsibilities of a team lead include:
# Coordinate team test efforts
# Update testing status to wiki ([bi]weekly)
# Ensure any defects are filed to block FedoraOnXo
# Establish and lead team irc meetings
# Stub test guidelines on wiki ... and coordinate w/ test team


* Suspend/resume
{{admon/note|NOTE|it's not expected that the team lead should create all the content, more that you provide some basic guidance so that the team can help fill in the blanks.}}
* Hibernate
** close lid
** low battery
* Use and report findings from ''powertop''
** Measure battery usage?
* Display brightness configuration (dim when idle)
* Remove/add power adapter while running
* Laptop Power/battery indicators behave as expected


== [[QA/TestPlans/Networking | Networking]] ==
== Team Member ==


* Wireless networking
As a team member, you are encouraged to:
** laptop wifi status indicator behaves as expected
# Provide suggested names for your team :)
** firmware loaded by NM
# Attend team meetings
** can connect using various methods (None/WEP/WPA)
# Contribute test plan guidance to the wiki for your team
** VPN?
# Notify your team lead of any blocking defects
** Sharing an ad-hoc network w/ another XO


== [[QA/TestPlans/Display | Display]] ==
= Test Areas =
 
* xrandr rotation?
* Using expected driver and resolution settings
* Changing vt's back and forth
* Coming out of suspend?
* touchpad interface?
* gamepad interface?
* Brightness up/down accelerator key?
* plymouth graphical boot enabled+working by default?
 
== [[QA/TestPlans/Audio | Audio]] ==
 
* pulse audio default settings (not muted)
* various mixer settings visible/editable (ala ''gnome-volume-control'')
* gnome system sounds enabled
* Recording (ala ''gnome-sound-recorder'')
* headphone jack / internal speaker
* Volume up/down accelerator key?
 
== [[QA/TestPlans/XOPerformance | Performance]] ==
 
The XO is a '''low''' resource environment ... this group will target identifying services and methods to improve system performance.
 
Activities include:
* Identifying/testing various service configuration settings
* Making recommendations for Live CD spin kickstart changes to reflect findings
* Kernel tunables?
* Recommended SD installation options?
* Recommended boot-time options?
* Improved desktop performance?
 
== Other ==


* test USB port
Testing of Fedora on the XO is broken out into several high-level groups with a focus on targeting the hardware environment of the XO.
* Anything specific to boot processing (plymouth noted above)
* Default configured services?


= Responsibilities =
The list of test areas includes:
 
* [[QA/TestPlans/XO Performance | Performance]]
This section outlines the test teams and various community points of contact.
* [[QA/TestPlans/Camera | Built-in Camera]]
 
* [[QA/TestPlans/Power Management | Power Management]]
* [[User:gdk]]
* [[QA/TestPlans/Networking | Networking]]
* [[User:katzj]]
* [[QA/TestPlans/Display | Display]]
* [[User:jlaska]]
* [[QA/TestPlans/Audio | Audio]]
 
* [[QA/TestPlans/XFCE | XFCE]]
Each test area is being split up for a different team of experts to deep dive into.  The table below will be used to outline the teams and their focus areas.
 
{|
! Test Area !! Team Name !! Team Members
|-
| Example: Orbital Laser
| Team Dunder Mifflin
| Jim, Pam, Stanley
|-
| Built-in Camera
|
|
|-
| Power Management
|
|
|-
| Networking
|
|
|-
| Display
|
|
|-
| Audio
|  
|
|-
|}


= HOWTO's =
= HOWTO's =
Line 172: Line 102:


For convenience, you can file an new Fedora XO bug by [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&blocked=FedoraOnXO clicking here].
For convenience, you can file an new Fedora XO bug by [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&blocked=FedoraOnXO clicking here].
{{admon/tip|New to bugzilla?|General information and guidance can be found at [[BugsAndFeatureRequests]].}}


== Triage NEW Fedora XO Bugs ==
== Triage NEW Fedora XO Bugs ==
Line 214: Line 146:
= Documentation =
= Documentation =


== Getting Started ==
{{admon/important|FIXME|need more docs!}}
* What are we trying to accomplish?
** According to [[OLPC/Fedora_on_XO]], allow users to run a standard Linux desktop, based on Fedora 10, on their XO system.
* What are the requirements for testing?
** List matching hardware
** Known issues on older XO's?
* How to sign-up for a team?
* How to communicate issues?
* Where to go for help?
** fedora-test-list
** #fedora-qa


* Where are test images posted?


== Before you get an XO ==
== Before you get an XO ==
Line 240: Line 154:
* Download the OS images mentioned in the sections below.
* Download the OS images mentioned in the sections below.


* Purchase a Sandisk Extreme III SD card SD card if you do not already have one
* Purchase a Sandisk Extreme III SD 4GB SD card if you do not already have one


== When you get an XO ==
== When you get an XO ==
Line 259: Line 173:
## Download key using instructions provided  
## Download key using instructions provided  
# Reboot
# Reboot
== Download a Fedora Live Image ==
Requirements:
* A computer with a internet connection
* A torrent client installed
First, determine which image you wish to load:
{|
! Build Date !! URI !! Announcement
|-
| Nov 4, 2008 || [http://torrent.fedoraproject.org/torrents//Fedora-10-Preview-i686-Live.torrent Fedora-10-Preview-i686-Live.torrent] || See [http://www.redhat.com/archives/fedora-devel-announce/2008-November/msg00001.html fedora-devel-announce]
|-
| Oct 23, 2008 || [http://torrent.fedoraproject.org/torrents//Fedora-10-Snap3-i686-Live.torrent Fedora-10-Snap3-i686-Live.torrent] || See [https://www.redhat.com/archives/fedora-devel-announce/2008-October/msg00015.html fedora-devel-announce]
|-
| Oct 17, 2008 || [http://torrent.fedoraproject.org/torrents/Fedora-10-Snap2-i686-Live.torrent Fedora-10-Snap2-i686-Live.torrent] || See [https://www.redhat.com/archives/fedora-devel-announce/2008-October/msg00012.html fedora-devel-announce]
|-
| Oct 10, 2008 || [http://torrent.fedoraproject.org/torrents/F10-Snap1-i686-Live.torrent F10-Snap1-i686-Live.torrent] || See [https://www.redhat.com/archives/fedora-devel-announce/2008-October/msg00006.html fedora-devel-announce]
|-
| Oct 1, 2008 || [http://katzj.fedorapeople.org/olpc/olpc-gnome.iso olpc-gnome.iso] || See [http://katzj.livejournal.com/439606.html katzj blog]
|-
|}
Fedora 10 snapshot images are provided by bittorrent and available at http://torrent.fedoraproject.org.  To download a torrent, you must:
Using the [http://www.transmissionbt.com transmission] client:
# Install the client: <pre> yum install transmission</pre>
# Download the torrent file.  For example, to download the torrent file for F10 Snapshot2 you might type: <pre>wget http://torrent.fedoraproject.org/torrents/Fedora-10-Snap3-i686-Live.torrent</pre>
# Download the snapshot by typing: <pre># transmissioncli Fedora-10-Snap3-i686-Live.torrent</pre>
Using the [http://www.bittorrent.com bittorrent-curses] client:
# Install the client: <pre> yum install bittorrent</pre>
# Download the torrent file.  For example, to download the torrent file for F10 Snapshot2 you might type: <pre>wget http://torrent.fedoraproject.org/torrents/Fedora-10-Snap3-i686-Live.torrent</pre>
# Download the snapshot by typing: <pre># bittorrent-curses Fedora-10-Snap3-i686-Live.torrent</pre>


== Installing your SD card ==
== Installing your SD card ==


Requirements:  
Requirements:  
* Another computer to flash the SD card
* Another linux computer to flash the SD card - the installation scripts require a linux machine
* SD reader
* SD card - specifically a Sandisk Extreme III SD card ([http://www.sandisk.com/Products/ProductInfo.aspx?ID=2348 4 Gig] capacity). The card must be 4Gb or larger. The installed image is 3.0 Gb + 576Mb
* SD card - specifically a Sandisk Extreme III SD card (either [http://www.sandisk.com/Products/Item(1973)-SDSDX3-2048-SanDisk_Extreme_III_SD_2GB.aspx 2 Gig] or [http://www.sandisk.com/Products/ProductInfo.aspx?ID=2348 4 Gig] capacity).
* SDHC reader - older SD card readers will not recognize the 4Gb SDHC card
* An internet connection
* An internet connection


To install a live image to your SD card:
To install a live image to your SD card:
# Make sure the ''syslinux'' package is installed: <pre># yum install syslinux</pre>
# Download a live image as described in [[QA/TestPlans/Fedora10_On_XO#Download_a_Fedora_Live_Image]]
# Download the installation script: <pre># wget http://katzj.fedorapeople.org/olpc/livecd-iso-to-disk.sh</pre>
# Download the installation script: <pre># wget http://katzj.fedorapeople.org/olpc/livecd-iso-to-disk.sh</pre>
# Download an OLPC Fedora Live image from: <pre># wget http://katzj.fedorapeople.org/olpc/olpc-gnome.iso</pre>
# Make sure you know what device your SD card is attached to.  Insert the SD card into the system and type: <pre># mount | grep media</pre>
# Write the live image to your SD card: <pre># bash livecd-iso-to-disk.sh --xo --extra-kernel-args 3 /tmp/olpc-gnome.iso /dev/sdb1</pre>
#* You need both the mount point (the part after /media/), and the device (the part after /dev/). If you get more than one line of output make sure you know what's attached to what.
# Make sure the SD card is not mounted: <pre># umount /media/usbdisk</pre>
#* Replace usbdisk with the mount point of your SD card.
# Write the live image to your SD card: <pre># bash livecd-iso-to-disk.sh --xo --overlay-size-mb 320 --swap-size-mb 256 Fedora-10-Snap3-i686-Live/F10-Snap3-i686-Live.iso /dev/sdb1</pre>
#* Replace sdb1 with the device name of your SD card.


{{admon/tip|Common Errors|For help resolving common errors found when writing your live image, see [[FedoraLiveCD/USBHowTo#Errors_and_Solutions]].
{{admon/tip|Common Errors|For help resolving common errors found when writing your live image, see [[FedoraLiveCD/USBHowTo#Errors_and_Solutions]].
Line 279: Line 234:


which will set the disk label to "msdos".
which will set the disk label to "msdos".
If you have any difficulty writing the image to the SD card, you should start by reformatting it using the instructions provided at [[FedoraLiveCD/USBHowTo#How_to_reformat_flash_drive]].


}}
}}
Line 285: Line 242:


* [http://wiki.laptop.org/go/Support_FAQ OLPC Support FAQ]
* [http://wiki.laptop.org/go/Support_FAQ OLPC Support FAQ]
[[Category:XO]]

Latest revision as of 20:49, 6 July 2009

Introduction

This document describes the test process by which Fedora 10 will be verified for operation consistent to that of other common laptop hardware. (For information regarding Fedora 11 and beyond on XO hardware, see OLPC/Fedora_on_XO.)

The goals of this plan are to:

  • Organize a community test effort
  • Communicate the strategy, scope and priorities of the planned tests to all relevant stake-holders for their input and approval
  • Serve as a base for the test planning for future Fedora releases
FIXME - Outline exit criteria
What bugs should be prioritized by development? How do we know if something should be fixed or not?

Strategy

In an effort to focus testing on specific XO hardware, the test strategy will be to:

  1. Define a set of high-level test areas that group similar functional components together
  2. Organize volunteers into teams, each team focused on a specific test area
  3. Teams will develop a rough test plan for their test area. Each team can dictate the required level of testing (both in terms of depth and breadth)

As no centralized automation is available, test teams are encouraged to leverage exploratory testing and built-in unit-tests where possible.

Use existing tools to facilitate writing new test builds to the XO's SD card (see livecd-iso-to-disk.sh).

Schedule

An up-to-date schedule for Fedora 10 is available at Releases/10/Schedule. A summary of Fedora XO related test milestones is noted below.

Date Milestone
Tue 2008-09-30 Public beta
Fri 2008-10-10 Snapshot#1
Fri 2008-10-17 Snapshot#2
Fri 2008-10-24 Snapshot#3
Tue 2008-11-04 Preview release
Tue 2008-11-25 General Availability

Scope

The scope of all testing should be representative of the XO hardware under test. No need to test Gigabit ethernet if the XO doesn't provide such an adapter. See the test area break down for more details.

Items not tested:

  • Installation of Fedora on an XO - only live image booting from the SD is planned for Fedora 10

Responsibilities

Test Teams

This section outlines the test teams and various community points of contact. If you are an XO Tester, please add your name to one of the groups on the XO Test Roll Call.

Team Lead

Responsibilities of a team lead include:

  1. Coordinate team test efforts
  2. Update testing status to wiki ([bi]weekly)
  3. Ensure any defects are filed to block FedoraOnXo
  4. Establish and lead team irc meetings
  5. Stub test guidelines on wiki ... and coordinate w/ test team
NOTE
it's not expected that the team lead should create all the content, more that you provide some basic guidance so that the team can help fill in the blanks.

Team Member

As a team member, you are encouraged to:

  1. Provide suggested names for your team :)
  2. Attend team meetings
  3. Contribute test plan guidance to the wiki for your team
  4. Notify your team lead of any blocking defects

Test Areas

Testing of Fedora on the XO is broken out into several high-level groups with a focus on targeting the hardware environment of the XO.

The list of test areas includes:

HOWTO's

The following sections are intended to provide high-level instructions for common tasks encountered by a Fedora XO tester.

File Fedora XO bugs

All bugs will be tracked in Bugzilla.

Please file all bugs found while testing Fedora on the XO with the following parameters:

  • product = Fedora
  • version = rawhide
  • blocks = FedoraOnXO

For convenience, you can file an new Fedora XO bug by clicking here.

New to bugzilla?
General information and guidance can be found at BugsAndFeatureRequests.

Triage NEW Fedora XO Bugs

To help with triaging ...

  1. Start with the list of NEW Fedora XO bugs at http://tinyurl.com/52wclh

For each bug, ask yourself the following questions:

  1. Is the bug specific to the XO hardware or environment?
  2. Was the following information provided?
    • Build tested (rawhide-YYYYMMDD or package versions)
    • Clear steps to reproduce the problem

If the above requirements have been met, please:

  1. Mark the bug as blocking 461806 (FedoraOnXO)
  2. Change to bug state to ASSIGNED

For further reading, the bug triage process is described in more detail at BugZappers.

Test a MODIFIED Fedora XO Bug

FIXME - provide tinyurl link to MODIFIED Fedora XO Bugs

Ask the following questions:

  1. Are you able to confirm that the steps to reproduce no longer trigger the failure?
  2. Have any new bugs been introduced as a result of the fix?
  3. Are any release notes required?

When comfortable with the fix:

  1. Add a comment indicating which build or software packages were tested
  2. Change the bug state to CLOSED with a resolution of RAWHIDE

If the reported issue has not been addressed:

  1. Add a comment indicating which build or software packages were tested that still exhibit the failure
  2. Change the state to ASSIGNED

Documentation

Before you get an XO

  • Download the OS images mentioned in the sections below.
  • Purchase a Sandisk Extreme III SD 4GB SD card if you do not already have one

When you get an XO

  1. Try out the hardware (sound should play at boot, webcam should take a picture)
  2. Update the Sugar OS with the "general public" clean-install procedure
  3. Boot the XO and connect to a nearby wireless network
    • To find OLPC mac address ... press <ctrl><alt><Neighborhood>. Then type
      ip addr list eth0
    • You may need to manually configure your SSID with
      iwconfig eth0 mode managed essid MYSSID
  4. Go to Software update in the Control Panel and apply any updates
  5. Obtain a developer key (see http://wiki.laptop.org/go/Developer_key#Getting_a_developer_key_for_your_running_XO_laptop)
    1. Load the Browse activity
    2. Navigate to file:///home/.devkey.html
      • Follow the instructions on the page
    3. Wait for developer key activation (more information at https://www.redhat.com/archives/fedora-test-list/2008-October/msg00229.html)
    4. Download key using instructions provided
  6. Reboot

Download a Fedora Live Image

Requirements:

  • A computer with a internet connection
  • A torrent client installed

First, determine which image you wish to load:

Build Date URI Announcement
Nov 4, 2008 Fedora-10-Preview-i686-Live.torrent See fedora-devel-announce
Oct 23, 2008 Fedora-10-Snap3-i686-Live.torrent See fedora-devel-announce
Oct 17, 2008 Fedora-10-Snap2-i686-Live.torrent See fedora-devel-announce
Oct 10, 2008 F10-Snap1-i686-Live.torrent See fedora-devel-announce
Oct 1, 2008 olpc-gnome.iso See katzj blog

Fedora 10 snapshot images are provided by bittorrent and available at http://torrent.fedoraproject.org. To download a torrent, you must:


Using the transmission client:

  1. Install the client:
     yum install transmission
  2. Download the torrent file. For example, to download the torrent file for F10 Snapshot2 you might type:
    wget http://torrent.fedoraproject.org/torrents/Fedora-10-Snap3-i686-Live.torrent
  3. Download the snapshot by typing:
    # transmissioncli Fedora-10-Snap3-i686-Live.torrent

Using the bittorrent-curses client:

  1. Install the client:
     yum install bittorrent
  2. Download the torrent file. For example, to download the torrent file for F10 Snapshot2 you might type:
    wget http://torrent.fedoraproject.org/torrents/Fedora-10-Snap3-i686-Live.torrent
  3. Download the snapshot by typing:
    # bittorrent-curses Fedora-10-Snap3-i686-Live.torrent

Installing your SD card

Requirements:

  • Another linux computer to flash the SD card - the installation scripts require a linux machine
  • SD card - specifically a Sandisk Extreme III SD card (4 Gig capacity). The card must be 4Gb or larger. The installed image is 3.0 Gb + 576Mb
  • SDHC reader - older SD card readers will not recognize the 4Gb SDHC card
  • An internet connection

To install a live image to your SD card:

  1. Make sure the syslinux package is installed:
    # yum install syslinux
  2. Download a live image as described in QA/TestPlans/Fedora10_On_XO#Download_a_Fedora_Live_Image
  3. Download the installation script:
    # wget http://katzj.fedorapeople.org/olpc/livecd-iso-to-disk.sh
  4. Make sure you know what device your SD card is attached to. Insert the SD card into the system and type:
    # mount | grep media
    • You need both the mount point (the part after /media/), and the device (the part after /dev/). If you get more than one line of output make sure you know what's attached to what.
  5. Make sure the SD card is not mounted:
    # umount /media/usbdisk
    • Replace usbdisk with the mount point of your SD card.
  6. Write the live image to your SD card:
    # bash livecd-iso-to-disk.sh --xo --overlay-size-mb 320 --swap-size-mb 256 Fedora-10-Snap3-i686-Live/F10-Snap3-i686-Live.iso /dev/sdb1
    • Replace sdb1 with the device name of your SD card.
Common Errors
For help resolving common errors found when writing your live image, see FedoraLiveCD/USBHowTo#Errors_and_Solutions.

If you get the error "Error: The flag 'boot' is not available for loop disk labels" when running parted. To fix this run:

syslinux -s /dev/sdb1

which will set the disk label to "msdos".

If you have any difficulty writing the image to the SD card, you should start by reformatting it using the instructions provided at FedoraLiveCD/USBHowTo#How_to_reformat_flash_drive.

FAQ