From Fedora Project Wiki
(More work to help visualize rhe's proposed changes)
m (Add questions section)
 
(39 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Key ==
{{header|qa}}
{{draft}}


See the table below for a sample format for test results.
This page will outline criteria for success for a Fedora AMI release


{| class="wikitable sortable" border="0"
= Release Criteria =
! Test Result !! Explanation
|-
| {{result|none}}
| ''Untested'' - This test has not been run, and is available for anyone to contribute feedback.
|-
| {{result|pass|liam}}
| ''Passed'' - The test has been run and the tester determine the test met the ''expected results''
|-
| {{result|inprogress|adamwill}}
| ''Inprogress'' - An inprogress result is often used for tests that take a long time to execute.  Inprogress results should be temporary and change to pass, fail or warn.
|-
| {{result|fail|jlaska|XYZ|ZXY}}
| ''Failed'' - Indicates a failed test.  A link to a bug must be provided.  See [[Template:Result]] for details on providing bug information. <references/>
|-
| {{result|warn|rhe}} <ref>Brief description about the warning status</ref>
| ''Warning'' - This test completed and met the ''expected results'' of the test, but other issues were encountered during testing that warrant attention.<references/>
|-
| {{result|pass|rhe}} {{result|warn|kparal}}
| ''Multiple results'' - More people can easily provide results to a single test case.
|-
| style="background:lightgrey;"| {{testresult/none}} {{result|none}}
| ''Unsupported'' - An unsupported test or configuration.  No testing is required.
|-
|}


== Priority ==
In addition to the existing [[Fedora Release Criteria]], the following criteria apply to EC2 Fedora images.  A <span style="color:green; font-weight:bold;">good</span> AMI '''must'''...
# allow non-root user login
# have SELinux enabled and ''enforcing'' by default
# not contain any package dependency problems
# not include any packages built outside of Fedora infrastructure


The explanation of test case priority is available at [[QA:Fedora_13_Install_Test_Plan#Test_Priority]].
= How to Test =


== Test Matrix ==
To ''create'' an EC2 image, one needs...
# an amazon account
# the {{package|python-boto}} libraries installed
# a python script to tell Amazon to build an instance
# the ability to SSH into a machine to run tests, etc.


{| class="wikitable sortable" border="1"
= Open Questions =
! Release Level !! Variation !! Test Area !! Test Case !! i386 !! x86_64 !! References
# Who is responsible for creating and building EC2 images?
|-
#:  
| Alpha
 
| boot.iso
= Additional Reading =
| Image Sanity
* [[Publishing_image_to_EC2]]
| [[QA:Testcase_Mediakit_ISO_Size]]
* [[Cloud_SIG]]
| {{result|none}}
* [[Cloud_SIG/EC2_Images]]
| {{result|none}}
| <references/>
|-
| Alpha
| boot.iso
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Checksums]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| boot.iso
| Boot Methods
| [[QA/TestCases/BootMethodsBootIso]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| boot.iso
| Image Source
| [[QA/TestCases/ImageSourceBootIso]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| boot.iso
| Install Source
| [[QA/TestCases/InstallSourceHttp]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| boot.iso
| Package Set
| [[QA/TestCases/PackageSetsDefaultPackageInstall]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Size]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Checksums]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Image Sanity
| [[QA:Testcase_Mediakit_Repoclosure]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Image Sanity
| [[QA:Testcase_Mediakit_FileConflicts]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Boot Methods
| [[QA/TestCases/BootMethodsDvd]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Image Source
| [[QA/TestCases/ImageSourceDvd]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Install Source
| [[QA/TestCases/InstallSourceDvd]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| DVD iso
| Package Set
| [[QA/TestCases/PackageSetsDefaultPackageInstall]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| CD iso
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Size]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| CD iso
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Checksums]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| CD iso
| Image Sanity
| [[QA:Testcase_Mediakit_Repoclosure]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| CD iso
| Image Sanity
| [[QA:Testcase_Mediakit_FileConflicts]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| CD iso
| Boot Methods
| [[QA/TestCases/BootMethodsCdrom]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| CD iso
| Image Source
| [[QA/TestCases/ImageSourceCdrom]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| CD iso
| Install Source
| [[QA/TestCases/InstallSourceCdrom]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| CD iso
| Package Set
| [[QA/TestCases/PackageSetsDefaultPackageInstall]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Live ISO
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Size]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Live ISO
| Image Sanity
| [[QA:Testcase_Mediakit_ISO_Checksums]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Live ISO
| Image Sanity
| [[QA:Testcase_Mediakit_Repoclosure]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Live ISO
| Image Sanity
| [[QA:Testcase_Mediakit_FileConflicts]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Live ISO
| Install Source
| [[QA:TestCases/Install Source Live Image]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| PXE images
| Boot Methods
| [[QA/TestCases/BootMethodsPxeboot]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| Partitioning
| [[QA/TestCases/PartitioningUninitializedDisks]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Partitioning
| [[QA:Testcase_Anaconda_autopart_install]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Partitioning
| [[QA:Testcase_Anaconda_autopart_(encrypted)_install]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Partitioning
| [[QA:Testcase_Anaconda_autopart_(shrink)_install]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Partitioning
| [[QA:Testcase_Anaconda_autopart_(use_free_space)_install]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Partitioning
| [[QA/TestCases/PartitioningExt4OnNativeDevice]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Partitioning
| [[QA/TestCases/PartitioningExt3OnNativeDevice]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Partitioning
| [[QA/TestCases/PartitioningExt2OnNativeDevice]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Partitioning
| [[QA/TestCases/PartitioningNoSwap]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Partitioning
| [[QA:Testcase_Partitioning_On_Software_RAID]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| User Interface
| [[QA:Testcase_Anaconda_User_Interface_Graphical]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| User Interface
| [[QA:Testcase_Anaconda_User_Interface_Text]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| User Interface
| [[QA:Testcase_Anaconda_User_Interface_VNC]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| User Interface
| [[QA:Testcase_Anaconda_User_Interface_Cmdline]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| User Interface
| [[QA:Testcase_Anaconda_User_Interface_serial_console]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| User Interface
| [[QA:Testcase_Anaconda_User_Interface_Telnet]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| User Interface
| [[QA:Testcase_Anaconda_rescue_mode]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Install Source
| [[QA/TestCases/InstallSourceFtpNonAnonymous]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Install Source
| [[QA/TestCases/InstallSourceNfs]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Install Source
| [[QA/TestCases/InstallSourceNfsIso]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Install Source
| [[QA/TestCases/InstallSourceHardDrive]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Repository
| [[QA:Testcase Additional Ftp Repository]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Repository
| [[QA:Testcase Additional Http Repository]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Repository
| [[QA:Testcase Additional Mirrorlist Repository]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Repository
| [[QA:Testcase Additional NFS Repository]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Memory Test
| [[memtest86+]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Boot Methods
| [[QA/TestCases/BootMethodsXenParaVirt]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Boot Methods
| [[QA/TestCases/BootMethodsKVM]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Boot Methods
| [[QA:Testcase_Boot_Methods_efidisk.img]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Kickstart Delivery
| [[QA/TestCases/KickstartKsHttpServerKsCfg]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Kickstart Delivery
| [[QA/TestCases/KickstartKsFilePathKsCfg]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Kickstart Delivery
| [[QA/TestCases/KickstartKsHdDevicePathKsCfg]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Kickstart Delivery
| [[QA/TestCases/KickstartKsNfsServerPathKsCfg]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Recovery
| [[QA:Testcase_Anaconda_updates.img_via_URL]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Recovery
| [[QA:Testcase_Anaconda_updates.img_via_installation_source]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Recovery
| [[QA:Testcase_Anaconda_updates.img_via_local_media]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Recovery
| [[QA:Testcase Anaconda save traceback to remote system]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Recovery
| [[QA:Testcase Anaconda save traceback to bugzilla]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Recovery
| [[QA:Testcase Anaconda save traceback to disk]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Recovery
| [[QA:Testcase Anaconda traceback debug mode]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Storage Devices
| [[QA:Testcase_install_to_SATA_device]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Storage Devices
| [[QA:Testcase_install_to_SCSI_device]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Alpha
| General
| Storage Devices
| [[QA:Testcase_install_to_PATA_device]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Storage Devices
| [[QA:Testcase_Install_to_BIOS_RAID]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Storage Devices
| [[QA:Testcase_Install_to_Hardware_RAID]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Final
| General
| Storage Devices
| [[QA:Testcase_Anaconda_ISCSI_No_Authentication]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Upgrade system
| [[QA:Testcase_Anaconda_Upgrade_New_Bootloader]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Upgrade system
| [[QA:Testcase_Anaconda_Upgrade_Skip_Bootloader]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Upgrade system
| [[QA:Testcase_Anaconda_Upgrade_Update_Bootloader]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Upgrade system
| [[QA:Testcase_Anaconda_Upgrade_Encrypted_Root]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Upgrade system
| [[QA:Testcase_Preupgrade]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
| Beta
| General
| Upgrade system
| [[QA:Testcase_Preupgrade_low_/boot_disk_space_to_install]]
| {{result|none}}
| {{result|none}}
| <references/>
|-
|}

Latest revision as of 17:19, 9 March 2011

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

This page will outline criteria for success for a Fedora AMI release

Release Criteria

In addition to the existing Fedora Release Criteria, the following criteria apply to EC2 Fedora images. A good AMI must...

  1. allow non-root user login
  2. have SELinux enabled and enforcing by default
  3. not contain any package dependency problems
  4. not include any packages built outside of Fedora infrastructure

How to Test

To create an EC2 image, one needs...

  1. an amazon account
  2. the python-boto libraries installed
  3. a python script to tell Amazon to build an instance
  4. the ability to SSH into a machine to run tests, etc.

Open Questions

  1. Who is responsible for creating and building EC2 images?

Additional Reading

Retrieved from "https://fedoraproject.org/w/index.php?title=User:Jlaska/Draft3&oldid=225923"