User:Jlaska/Fedora 15 Installation automation ideas

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

This page is intended to outline the current state of automated installer tests, identify possible short-term objects and identify task owners.

Existing Installer Tests

This section lists tests or scripts that are used now during manual verification of the installer.

1. checkbot.sh - runs multiple tests
   1. Build test - runs ./autogen.sh && ./configure.sh && make
   2. Pylint test - Runs tests/pylint/runpylint.sh
2. tests/anaconda_storage - Storage unit testsuite in the clumens branch of autoqa.git
3. compose-tree script - Script that creates a mock chroot and creates install images using pungi (see also instructions)
4. tests/rats_install/install.py - AutoQA test script that does a basic automated kickstart install

Project Ideas

Update AutoQA storage unit testsuite

Scope

The patchset in the clumens autoqa branch may need to be refreshed to master and re-run to check for proper operation. This idea involves rebasing the patch to master, and re-running the tests to ensure they continue to work. In addition, we need to add more storage tests.

Skill Level

Advanced

Owner

Chris Lumens

Status

- Code complete, pending merge into master (see autoqa clumens branch)

Create a bugzilla query (or script) to identify storage unit testsuite candidates

Scope

Work with anaconda-devel to identify a pattern to identify bug reports that may have traceback information that can be used to craft new storage unit tests. This task does not involve creating the tests, only identifying a bugzilla query or script that can query bugzilla for test candidates.

Skill Level

Beginner

Owner

unassigned

Create AutoQA test based on checkbot.sh script

Scope

Currently, anaconda-devel uses a script called checkbot.sh to run pylint and make against anaconda source. This script would be pulled into AutoQA as a new test. Eventually, the test would be run after git commits, but in the short-term, could run on a nightly basis.

Skill Level

Moderate

Owner

James Laska and Chris Lumens

Status

- code complete, pending push to master (see autoqa clumens branch)

Create AutoQA post-git-commit watcher

Scope

To support triggering tests based on git commits, create a new AutoQA watcher.

Skill Level

Moderate

Owner

- preliminary support exists in autoqa clumens branch

Automate Installation test matrix

Scope

Automate installation test matrix (see Is_anaconda_broken_proposal). This is definitely not a short-term deliverable, and is dependent on several tasks already on this page. Open to additional suggestions/ideas on short-term tasks that are in line with this larger goal.

Skill Level

Advanced

Owner

unassigned

Create AutoQA test based on compose-tree script

Scope

Create a new AutoQA test using the compose-tree script that runs pungi inside a mock chroot. This test is intended to run after a koji build of anaconda or it's dependent packages.

Skill Level

Moderate

Owner

unassigned

Verify RHBZ #576439 and integrate VirtIO into rats_install test

Scope

With RHBZ #576439, we now can monitor installation progress without requiring an active network to transmit logs using minimon. Installing without networking allows test automation that more accurately mimics non-network installations. This task involves testing RHBZ #576439 to confirm that the required support exists as well as updating rats_install/install.py to use virtio for installation progress monitoring.

Skill Level

Moderate

Owner

unassined

Update AutoQA test detection to handle subdirectories

Scope

Since various anaconda tests are being added now, it makes sense to organize them all as subdirectories of a tests/anaconda/ directory. To support this, AutoQA needs to be able to traverse down into directories and look for subdirectories containing tests.

Skill Level

Moderate

Owner

unassigned