From Fedora Project Wiki
General Information
Communication and Community
- how many projects are using it
- Igor is used in the oVirt Project for oVirt Node. Additionally internally by Red Hat and IBM.
- how old is it
- Approx. one year (1yr)
- how many active devs from how many orgs
- About one (1) dev, from Red Hat
- quality of docs
- Not bad, getting better
- how much mailing list traffic is there?
- No ML right now
- what is the bug tracker?
- RHBZ
- what is the patch process?
- Currently through gitorious merge requests (migration to github planned)
- what is the RFE process?
- RFE bug in bz
High level stuff
- how tightly integrated are the components
- What components?
- what license is the project released under
- GPL2+
- how much is already packaged in fedora
- igord is completely packaged
- igor client (needed to be part of th eos under test) is not yet packaged
API
- what mechanism does the api use (xmlrpc, json-rpc, restful-ish etc.)
- RESTful XML/JSON
- can you schedule jobs through the api
- yes
- what scheduling params are available through the api
- (testsuite, host, profile, [kargs]) tuple
Results
- how flexible is the schema for the built in results store
- A job consists of many steps (testcases) there can be one result for each step (testcase)
- each step (testcase) can additionally be annotated and artifacts (files) can be attached
- what data is stored in the default result
- igor has no persistent result store, results are lost after the daemon quits
- a hook mechanism is inteded to feed the results (available in xml/json) into the actual store (jenkins in the current case, via junit)
- common results available are: pass/fail, job informations, testsuite informations, host informations, artifacats, annotations
- is there a difference between failed execution and status based on result analysis
- could you rephrase this question?
- what kinds of analysis are supported
- some basic passed/failed analysis is done, notification of any followup/extrernal component is inteded to be realized by hooks
- currently there is a web ui, a junit output and a fancy cli application displaying the results
VM management
- does it work with any external systems (ovirt, openstack etc.)
- there is currently a livirt backend
- does it support rapid cloning
- not yet, but it's possible to add
- how are vms configured post-spawn
- vms are only configured through kernel arguments (a kernel argument can spawn a kickstart based instalation, then the vm can additionally be configured using thes kickstart file)
- control over vm configuration (vnc/spice, storage type etc.)
- yes, through the API
- ephemeral client support?
- it's a key feature of igor
- volatile and "persistent" VMs are supported as well as real hosts
Test harness
- base language
- simple harness written in python, with a bash wrapper exposing all py functions
- how tightly integrated is it with the system as a whole
- makes some basic assumptions about the base os (vcs, uinput, bash, ..)
- are any non-primary harnesses supported
- basically yes, but no other is yet provided, xpresserng is a candidate
- another party is working on autotest support
Test execution
- how are tests stored
- tests are executables (typical python or bash), stored in a filesystem
- the SUT retrieves them via the API
- support for storing tests in vcs
- yes, is done in ovirt-node
- method for passing data into test for execution
- yes, via API
- alternative is i nthe works
- how are parameters stored for post-failure analysis
- all parameters are kept in the job status (vcan be retrieved via API)
- support for replaying a test
- can re-run previous job
- can tests be executed locally in a dev env with MINIMAL setup
- yes, e.g. the script itself
- yes, using the libvirt-only configuration, where only libvirt is required and used to run the tests
- external log shipping?
- could you rephrase this question?
- any command can be run as long as it's availabel or distributed using the "lib" feature of igor
- how tightly integrated is result reporting
- result reporting is done using the API, the UT is using the API, as well as any client triggering jobs.
- the SUT can report results, add annotations, even trigger another job or add artifacts
- what kind of latency is there between tests?
- if anew VM is created for a job there is time to prepare it
- if a VM is re-used there are a couple of seconds of management work done in the background
- if a real host is used it really depends on how quickly the host boots.