From Fedora Project Wiki
No edit summary
Line 3: Line 3:
This page provides a high-level roadmap for implementing the [[Is_anaconda_broken_proposal]] project.  More detailed tasks can be found in [https://fedorahosted.org/autoqa/milestone/Automate%20installation%20test%20plan autoqa TRAC roadmap].  We follow these steps to define the methods by which we initiate testing
This page provides a high-level roadmap for implementing the [[Is_anaconda_broken_proposal]] project.  More detailed tasks can be found in [https://fedorahosted.org/autoqa/milestone/Automate%20installation%20test%20plan autoqa TRAC roadmap].  We follow these steps to define the methods by which we initiate testing


== Step 1:Setup test enviroment ==
* Virtualization
** python-fedora installed
** libvirt installed
** qemu-kvm installed
** python-virtinst installed
** pax installed
** service libvirtd started


*Logging
First, in order to provide a consistent and documented test approach, the existing Fedora Install test plan <ref>[[QA:Fedora_13_Install_Test_Plan]]</ref> will be revisited.  The test plan will be adjusted to ensure proper test coverage for the failure scenarios listed above.  Existing test cases will be reviewed for accuracy.  New test cases will be created using the [[Template:QA/Test_Case]] template.  Finally, the test plan will be adjusted to match the improved Fedora Release Criteria <ref>[[Fedora Release Criteria]]</ref>.  This includes adjusting the test case priority to match milestone criteria.
** service rsyslog started
** rsyslog listening on specific port


== Step 2: Identifying boot methods, preparing installation media and checking media sanity==
Next, in order to reduce the setup/execution time, improve efficiency and to provide test results on a more consistent basis, a subset of test cases will be chosen for automation. Tests will be written in python and will be developed and executed on a system supporting KVM virtualization.   Test scripts will be responsible for preparing a virtual install environment, initiating a kickstart install and validating the results. Once an initial batch of tests exist, they will be formally integrated into the [[AutoQA]] project.
{| class="wikitable"
! Method!! Files !! Sanity !! Implementation
|-
| style="background-color:green;" |URL
| style="background-color:green;" |initrd.img, vmlinuz
| style="background-color:green;" |sha256sum,md5sum,version
| style="background-color:green;" |url_install/url_sanity.py
|-
| style="background-color:#66FF00;" |DVD
| style="background-color:#66FF00;" |DVD.iso
| style="background-color:#66FF00;" |size,sha256sum, md5sum
| style="background-color:#66FF00;" |dvd_install/dvd_sanity.py
|-
| CD
| 4 CD.iso
| size,sha256sum, md5sum
| cd_install/cd_sanity.py
|-
| style="background-color:lightgreen;" |Minimal Boot Media
| style="background-color:lightgreen;" |boot.iso
| style="background-color:lightgreen;" |size,sha256sum, md5sum
| style="background-color:lightgreen;" |bootiso_install/bootiso_sanity.py
|-
| Live
| live.iso
| size,sha256sum, md5sum
| live_install/live_sanity.py
|}


== Step 3: Preparing kick start file ==
Last, a method will be developed for collecting test results into a single test result matrix.  Results may be posted to the wiki directly, or a custom turbogears application may be needed to display results <ref>For a similar project see [http://jlaska.fedorapeople.org/irb.png screenshot] of ''is rawhide broken'' and [http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=front-ends/israwhidebroken;hb=HEAD source code] and the [[QA:Rawhide_Acceptance_Test_Plan]].</ref>.  The results will be easily accessible for testers and the installer development team.
Most of the test cases can be covered by configuring different kick start files.
As the startup, the typical kick start template is choose to be implemented.


'''Installation Kick Start Options'''
<!-- Describe the scope of any project work or properties that will be affected by the proposal -->
{|
!!!Description !!Scripts!! Test Case
|-
|rowspan=3 width=100 |Partitioning
|width=200 |autopart
|width=500 |
<pre>
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel
# Disk partitioning information
autopart
</pre>
|width=300 |[[QA:Testcase Anaconda autopart install]]
|-
|autopart encrypted
|<pre>
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel
# Disk partitioning information
autopart  --encrypted --passphrase=redhat
</pre>
|[[QA:Testcase Anaconda autopart (encrypted) install]]
|-
|specify partition
|<pre>
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel
# Disk partitioning information
part / --fstype="ext4" --size=3000
part /boot --fstype="ext4" --size=500
part swap --fstype="swap" --size=500
</pre>
|
|-
| rowspan=4 |Install Source
| Http
|<pre>repo=http://download.fedora.redhat.com/pub/fedora/linux/development/i386/os</pre>
|[[QA/TestCases/InstallSourceHttp]]
|-
|Anonymous Ftp
|<pre>repo=ftp://download.fedora.redhat.com/pub/fedora/linux/development/i386/os</pre>
|[[QA/TestCases/InstallSourceFtpAnonymous]]
|-
|NonAnonymous Ftp
|<pre>repo=ftp://USER:PASS@download.fedora.redhat.com/pub/fedora/linux/development/i386/os</pre>
|[[QA/TestCases/InstallSourceFtpNonAnonymous]]
|-
|}


'''Upgrade Kick Start Options'''
The project will be divided into several phases.  
{|
! !!Description !!Scripts!! Test Case
|-
|rowspan=3 width=100 | Boot loader
|width=200 |Install new bootloader
|width=500 |
<pre>
# System bootloader configuration
bootloader --location=mbr
</pre>
|width=300 |[[QA:Testcase Anaconda Upgrade New Bootloader]]
|-
|Do not install a bootloader
|<pre>
# System bootloader configuration
bootloader --location=none
</pre>
|[[QA:Testcase Anaconda Upgrade Skip Bootloader]]
|-
|Upgrade Exiting boot loader
|<pre>
# System bootloader configuration
bootloader --location=none --upgrade
</pre>
|[[QA:Testcase Anaconda Upgrade Update Bootloader]]
|-
|}


'''Rescure Kick Start Options'''
'''Phase#1''' - ''proof of concept''
List test scenarios,it will make clear that which suite needs to
* {{result|pass|Revise Fedora Install test plan to ensure adequate test coverage exists for failure scenarios listed above}}
support.
* {{result|pass|Select a small, but representative, subset of test cases from the install test plan to automate}}
<!-- CSS Rotate
* {{result|pass|Create python scripts to prepare KVM-based virtual environments for testing, initiate kickstart installs, and validate results}}
<style type="text/css">
* {{result|pass|Investigate methods for leveraging GUI automation to aid in automating applicable test cases}}  
td.vertical-text {
color:#333;
border:0px solid red;
writing-mode:tb-rl;
-webkit-transform:rotate(270deg);
-moz-transform:rotate(270deg);
-o-transform: rotate(270deg);
white-space:nowrap;
display:block;
bottom:0;
width:20px;
height:20px;
font-family: ‘Trebuchet MS’, Helvetica, sans-serif;
font-size:24px;
font-weight:normal;
text-shadow: 0px 0px 1px #333;
}
</style>
-->


{| class="wikitable  collapsible collapsed" border="1" width="85%"
'''Phase#2''' - ''implementation''
! colspan="7"|Test Scenarios
* {{result|inprogress|Identify and automate remainder of test cases from the install test plan}}
|-
* Identify and integrate anaconda built-in unit tests into tests
! colspan="1" class="unsortable"| &nbsp;
* Identify test event triggers which will be used to automatically initiate testing.
! colspan="6" class="unsortable"|Boot Method <ref name="Boot Method"> Boot Method: http://docs.fedoraproject.org/en-US/Fedora/14/html/Installation_Guide/ch04s06.html </ref>
* Identify test cases where GUI automation will be required
|-
! colspan="1"|Test
! DVD
! CD
! boot.iso
! PXE images
! efidisk.img
! Live Image
<!--
|-
! colspan="1" class="unsortable"|Methods to load install.img (not applicable after F15)
! colspan="6" class="unsortable"|&nbsp;
|-
|stage2=cdrom
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|stage2=http://server/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|stage2=ftp://[user:passwd@]server/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|stage2=nfs:
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|stage2=hd:
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|askmethod
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:red; color:white;"|no
-->
|-
! colspan="1" class="unsortable"|Installation Method <ref>Installation Method: http://docs.fedoraproject.org/en-US/Fedora/14/html/Installation_Guide/s1-installmethod-x86.html</ref><ref name="bootoption">Anaconda Boot Options: http://fedoraproject.org/wiki/Anaconda/Options</ref>
! colspan="6" class="unsortable"|&nbsp;
|-
|autodetect cdrom
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=cdrom
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=http://host/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=ftp://[user:passwd@]server/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=nfs:server:/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=nfsiso:server:/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=hd:device/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=hdiso:
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|repo=livecd:
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:lightgreen;"|yes
|-
! colspan="1" class="unsortable"|Methods to locate kickstart file <ref name="bootoption"/>
! colspan="6" class="unsortable"|&nbsp;
|-
|ks (autodetect from dhcp ???(default is nfs))
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=http://server/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=ftp://[user:passwd@]server/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=nfs:server:/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=file:/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=hd:device:/path
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=bd:biosdevice:/path
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=floppy
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
|ks=cdrom
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:yellow;"|unknown
|-
! colspan="1" class="unsortable"|Methods to locate updates.img
! colspan="6" class="unsortable"|&nbsp;
|-
|updates
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|updates=http
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|images/updates.img
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|RHUpdates
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:red; color:white;"|no
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
! colspan="1" class="unsortable"|Methods to determine display device
! colspan="6" class="unsortable"|&nbsp;
|-
|graphical
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|-
|text
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|-
|vnc
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|cmdline
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|serial console=ttyS0
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:#66FF00;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|telnet
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
! colspan="1" class="unsortable"|Methods to choose kickstart network device <ref name="bootoption"/>
! colspan="6" class="unsortable"|&nbsp;
|-
|ksdevice=link
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:red; color:white;"|no
|-
|ksdevice=bootif
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|ksdevice=eth0
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|ksdevice=AA:BB:CC:DD:EE:FF
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
! colspan="1" class="unsortable"|Methods to activate network device
! colspan="6" class="unsortable"|&nbsp;
|-
|ip=dhcp
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:green;"|yes
|style="background-color:red; color:white;"|no
|-
|ip=a.b.c.d
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|dns=
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|gateway=
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|netmask=
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:lightgreen;"|yes
|style="background-color:red; color:white;"|no
|-
|}


== Step 4: Creating Virt Guest==
'''Phase#3''' - ''integration''
virt-install is adopted to initialize the guest. The functions to create guest with virt-install from url, DVD,ect. should be defined firstly.
* Create appropriate control files and test wrappers to allow for scheduling tests through AutoQA (see [[Writing_AutoQA_Tests]])
The following functions are defined in the class '''VirtGuest''' located in '''/lib/python/virtguest.py'''.
* Develop or update [[AutoQA]] test event hooks to accommodate new test events (see [[Writing_AutoQA_Hooks]])
{|
* Implement initial test result dashboard intended to eventually replace the wiki test matrix. The dashboard will also support FAS user test result submission. This will likely rely on
!Method!!Function
|-
|width=100|URL
|width= 200|url_create()
|-
|DVD,CD,boot.iso
|iso_create()
|}
 
 
 
Identify potential test drivers that trigger off of those events.
 
* URL install source available (typically, http://download.fedoraproject.org/pub/fedora/linux/development/15/i386/os)
** url_sanity.py (for provided {{filename|images/boot.iso}})
** url_install.py
** bootiso_install.py
** pxe_install.py (kind of silly, but a possible future test)
* DVD image(s) available (typically, somewhere on http://serverbeach1.fedoraproject.org/pub/alt/stage/)
** dvd_sanity.py
** dvd_install.py
** hddvc_install.py
** nfsdvc_install.py
* CD image(s) available (typically, somewhere on http://serverbeach1.fedoraproject.org/pub/alt/stage/)
** cd_sanity.py
** cd_install.py
** hdcd_install.py
** nfscd_install.py
 
==Step 5 : Logging and watching the steps of installation==
Get the installation output and logs at every step:
{|
! Stage !! Log file !! End label
|-
|width =100|Boot
|width=300|boot.log
|width=300|Greetings
|-
|Stage1
|stage1.log
|Running anaconda [version], system installer - please wait
|-
|Stage2
|stage2.log
|
|}
 
== Key ==
Set priorities for each test. The test drivers will support the tests with high priority.
 
{| class="wikitable sortable" border="0"
! Colour !! Explanation
|-
| style="background-color:green;"|yes
| supported and default (high priority)
|-
| style="background-color:#66FF00;"|yes
| supported (medium priority)
|-
| style="background-color:lightgreen;"|yes
| supported (low priority)
|-
| style="background-color:red; color:white;"|no
| unsupported configuration
|-
| style="background-color:yellow;"|unknown
| unknown
|-
|}
 
==References==
<references/>

Revision as of 10:47, 18 March 2011

This page provides a high-level roadmap for implementing the Is_anaconda_broken_proposal project. More detailed tasks can be found in autoqa TRAC roadmap. We follow these steps to define the methods by which we initiate testing


First, in order to provide a consistent and documented test approach, the existing Fedora Install test plan [1] will be revisited. The test plan will be adjusted to ensure proper test coverage for the failure scenarios listed above. Existing test cases will be reviewed for accuracy. New test cases will be created using the Template:QA/Test_Case template. Finally, the test plan will be adjusted to match the improved Fedora Release Criteria [2]. This includes adjusting the test case priority to match milestone criteria.

Next, in order to reduce the setup/execution time, improve efficiency and to provide test results on a more consistent basis, a subset of test cases will be chosen for automation. Tests will be written in python and will be developed and executed on a system supporting KVM virtualization. Test scripts will be responsible for preparing a virtual install environment, initiating a kickstart install and validating the results. Once an initial batch of tests exist, they will be formally integrated into the AutoQA project.

Last, a method will be developed for collecting test results into a single test result matrix. Results may be posted to the wiki directly, or a custom turbogears application may be needed to display results [3]. The results will be easily accessible for testers and the installer development team.


The project will be divided into several phases.

Phase#1 - proof of concept

  • Pass pass Revise Fedora Install test plan to ensure adequate test coverage exists for failure scenarios listed above
  • Pass pass Select a small, but representative, subset of test cases from the install test plan to automate
  • Pass pass Create python scripts to prepare KVM-based virtual environments for testing, initiate kickstart installs, and validate results
  • Pass pass Investigate methods for leveraging GUI automation to aid in automating applicable test cases

Phase#2 - implementation

  • Inprogress inprogress Identify and automate remainder of test cases from the install test plan
  • Identify and integrate anaconda built-in unit tests into tests
  • Identify test event triggers which will be used to automatically initiate testing.
  • Identify test cases where GUI automation will be required

Phase#3 - integration

  • Create appropriate control files and test wrappers to allow for scheduling tests through AutoQA (see Writing_AutoQA_Tests)
  • Develop or update AutoQA test event hooks to accommodate new test events (see Writing_AutoQA_Hooks)
  • Implement initial test result dashboard intended to eventually replace the wiki test matrix. The dashboard will also support FAS user test result submission. This will likely rely on
  1. ↑ QA:Fedora_13_Install_Test_Plan
  2. ↑ Fedora Release Criteria
  3. ↑ For a similar project see screenshot of is rawhide broken and source code and the QA:Rawhide_Acceptance_Test_Plan.
Retrieved from "https://fedoraproject.org/w/index.php?title=Is_anaconda_broken_roadmap&oldid=227647"