Motivation
Sometimes a test case is only relevant for specific environment. Test Case Relevancy allows to filter irrelevant test cases out.
Implementation
Test case relevancy is implemented using a function which takes environment parameters (see below for the complete list) and returns:
False— test case is not relevant for this environmentA=X B=Y C=Z— test case is relevant for the modified environment
The latter case allows to adjust the environment in which the test case is to be executed by providing the list of environment variables which will be passed to the test.
Syntax
Test case relevancy is defined by one or more condition: decision rules.
Allowed operators are: = == != < <= > >= &&. Anything beyond a # sign is considered to be a comment and will be ignored.
Everything is case insensitive. First matching rule wins (terminates immediately the relevancy evaluation, the rest of the rules will be ignored).
Defaults
If there is no rule specified, test case relevancy defaults to True, that is test case is relevant and should be executed.
Environment
Acceptable parameters defining the environment are:
- product — product name (rhel rhel-5 rhel-5.6 rhdts rhscl)
- distro — distribution (rhel-6 rhel-6.3 rhscl-1.0.0 f-28)
- variant — distro variant (Client Desktop Server Workstation)
- arch — architecture (i386 ppc64 s390x x86_64)
- collection — software collection (python27 python33 perl516...)
- component — component to be tested (php, apache, ... )
While distro is always used to define the operating system the software is supposed to run on, product is used to describe the target product subscribed and consumed by the customer (can be a layered product on top of RHEL such as RHSCL, or RHEL itself if the component is included directly in the operating system).
Distro comparison operates in two modes:
- Major mode — When comparing against a major version such as
distro < rhel-6other major versions are considered (matches any ofrhel-3 rhel-4 rhel-5). - Minor mode — If minor version is provided as well, for example
distro < rhel-6.3, comparison is performed only within the given major (matchesrhel-6.0 rhel-6.1 rhel-6.2).
Rules which contain environment parameters which are not known at the time of evaluation will be skipped.
Examples
mod_wsgi relevant for RHEL6 and newer:
distro < rhel-6: False
busybox not available for s390x on RHEL6:
arch = s390x && distro = rhel-6: False
perl-Config-General not present in the Client variant:
variant = Client: False
Run python unit tests under valgrind on suitable archs only:
arch = ia64: PHASES=novalgrind arch = s390x && distro < rhel-6: PHASES=novalgrind
New component python-ctypes added in rhel-5.8:
distro < rhel-5: False distro < rhel-5.8: False
Exim present solely in RHEL5:
distro != rhel-5: False