This page explains how to go about arranging a Test Day event. If you have any questions or concerns about the Test Day process, please contact the QA group.
Get people on board
Make sure all involved parties are aware of the Test Day. Usually, you would want the maintainer(s) of the affected component(s) to be aware of the Test Day and, ideally, present when it happens. If you know of experienced users of the component(s) who will likely be able both to provide useful feedback at the Test Day and bring in other users, it will help to bring them on board in advance, too. If the affected component has a triager - see the BugZappers team's list of components and triagers - bring them in too.
Set a date
Aim for a day when:
- Most involved people (organizer, maintainers, experienced users, triagers, QA team) can be available
- The feature will be close enough to complete for the testing to mean something, but outside of any deep related freeze periods
- No other event which would be likely to draw away users with an interest in the Test Day is happening - including other Test Days!
Try to be flexible when it comes to the time the Test Day actually happens: consider timezone issues, especially if many users of the feature are in a particular geographical area. Ideally, aim to have developers or experienced users present throughout the entire period when it is the date of the Test Day somewhere in the world.
Create the Wiki page
This will be the central source of information regarding your Test Day: make sure it's complete, accurate and up to date at all times. Use this Test Day template as a base: copy and paste the source of that page and fill it in for your event. Your page should be given a name of the form Test_Day:(DATE)_(TOPIC) - e.g. Test_Day:2009-05-07_Virtualization . It should be in the Test Days category. You can add your event to the Test Day schedule for the appropriate release. Release Test Day schedules are linked to from the main Test Days page.
As a goal, your Wiki page should provide enough information that someone could perform all the testing and provide their results without any other reference or help.
Create test cases
A test case is simply a precise set of instructions which cause a reliably repeatable operation to occur which allows you to determine some kind of information from the results of the operation. There is a template to be used for creating test cases: if you visit that page, you will see a link which explains how to use this template to create a test case. You can see example test cases in the test case category page. When creating a test case, try to make sure the instructions are explicit enough that anyone who may want to participate in the test day can follow them, and entirely unambiguous so that the user cannot perform the test incorrectly. Don't cram multiple operations into a single test case: create separate test cases for each different operation you wish to test. If possible, try to create the Test Case in such a way that it will be re-usable in the future (ask yourself if someone running an edition of Fedora from five years after you write the test case would still be able to follow it), although this is not always possible.
Link to each of the test cases from the How to Test section of the Wiki page.
Set up the results table
The basic format of the results table is a row for each tester (or test system), and columns for each test, along with a column for hardware information if appropriate and one for comments. It helps to provide an example row showing the format that should be used for filling in each column. It also helps to request that longer comments be added below the results table (or as separate pages) with a link from the table, to prevent the table from filling up with long comments and becoming difficult to read.
Build a live CD
If some or all of the test cases can be successfully performed from a live boot, it is highly recommended to provide a live CD image. Instructions on doing so can be found here. Remember to customize the spin to include whatever application you want to test, and any utilities that are needed for testing it. Also make these available through a repository if they're not in rawhide, for people who want to test from their rawhide system rather than the live CD. Put the live CD image up in a fedorapeople space; contact X to extend your quota if necessary.
Promote the Test Day
- Blog about it (and make sure your blog's on Planet Fedora)
- Mail fedora-test-list
- Contact the Fedora Weekly News QA beat author to get news about your event added to FWN. FWN is written over the weekend, so make sure to make contact before the weekend before your event
- If it affects a specific country, contact the Ambassador(s) for that country. If it affects a specific community, contact the places where that community gathers - IRC, forums, whatever
- If it's possibly of interest to a wider community than just Fedora users (make sure you have a live CD, for this one!), submit it to some general interest news sites
It's probably best to announce the test day about a week in advance, and do a 'quick reminder' post the day before.
Pre-event reminder
Remind the involved parties - maintainers, experienced users, triagers, QA team - about the event shortly before it happens, to make sure they show up. And remember to show up yourself!
Update Test_Day:Current
Update the Test_Day:Current wiki redirect. The Test_Day:Current page is used to conveniently direct testers to the most current test day. To update the page, login to the wiki and click here.
Do a post-event review
Doing a review of the Test Day once it's finished helps make it clear what the Test Day achieved, acts as a useful reference for issues discovered, and helps to promote the Test Day process to other users. A good way to write the review is to base it around a generated list of the bugs reported as a result of the Test Day. You can generate such a list by running this command:
grep show_bug\.cgi\?id=[0-9][0-9][0-9][0-9][0-9][0-9] testday.html -o | sort -u | cut -d= -f 2 | tr '\n' ',' | xargs bugzilla query --outputformat="%{bug_id} %{bug_status} %{resolution} - %{short_desc}" -b
where testday.html is a saved copy of the Test Day Wiki page. The command will produce a list of bugs whose bug report URLs are in the Test Day page, with status information on each bug. You may have to weed out any 'false' references to pre-existing bugs which are mentioned on the page for some reason. This command requires the 
python-bugzilla