User:Roshi/QA/Cloud Docs/what to test

Overview

This page is for a list of all the things the Cloud WG wants to test on each image.

What we have:

• EC2 Upload Testcase
• Openstack Upload
  • Some other related testcases (old)

What we want to test:

• Image uploads
• Image boots
• Services are running
  • ssh
  • install packages from repo
  • cloud-init features
    • install/update packages from a repo
    • ssh keys
    • user data script
    • host name set properly from metadata
  • firewall/root disabled
  • selinux enforcing
  • messages are visible on the serial console (including the ssh host key fingerprint)
  • test behavior when launched into a larger-than-required volume (growroot?)
  • sudo works properly (and specifically for the fedora user, not arbitrary accounts)
  • external config tools work properly (vagrant, ansible, etc.) -- mostly mentioned due to past sudo issues with requiretty
  • ?
• Image content sanity checks
  • On-disk size is under some threshold (suggest 500M -- we are at 484M now with F21 rawhide)
  • qcow2 size is under some threshold (suggest 200M -- it's a magic number. We are at 187M now.)
  • libX11 not included (as a dependency explosion "canary")
  • only one version of python (version 2 or 3) is included

• Docker?
• Big Data?

How?

• What test-scripts do we have currently?
• Does everything need to be done from scratch?

Basic httpd test:

• launch image with httpd-allowed security group
  • cloud-init script automates:
    • install httpd
    • put some static content with known string into place
    • start httpd
  • test that known string is accessible via http

Wiki Test Cases

• QA:Testcase_Cloud_ssh
• QA:Testcase_Cloud_package_installation
• QA:Testcase_Cloud_package_upgrade
• QA:Testcase_Cloud_firewall_disabled
• QA:Testcase_Cloud_root_disabled
• QA:Testcase_Cloud_selinux
• QA:Testcase_Cloud_serial_console
• QA:Testcase_Cloud_sudo
• QA:Testcase_Cloud_external_configuration_tools