A test suite is a collection of tests. QMTest can run an entire test suite at once, so by grouping tests together in a test suite, you make it easier to run a number of tests at once. A single test can be a member of more than one test suite. A test suite can contain other test suites; the total set of tests in a test suite includes both those tests included directly and those tests included as part of another test suite. Every test suite has a name, following the same conventions given above for tests.
One use of test suites is to provide groups of tests that are run in
different situations. For example, the nightly test
suite might consist of those tests that should be run automatically every
night, while the checkin test suite might consist of
those tests that have to pass before any changes are made to the target
application.
Section 1, “Tests” explains how you may arrange
tests in a tree hierarchy, using a period
(“.”) as the path separator in test
names. QMTest defines an implicit
test suite for each directory. The name of these implicit
test suites is the same as the name of the directory. The implicit test
suite corresponding to a directory contains all tests in that directory
or its subdirectories.
Let us create some more tests, but this time within their respective subdirectories:
>qmtest create --id=dir1.one -a expression='True' test python.ExecTest>qmtest create --id=dir1.two -a expression='False' test python.ExecTest>qmtest create --id=dir2.one -a expression='True' test python.ExecTest>qmtest create --id=dir2.dir3.one -a expression='False' test python.ExecTest>qmtest create --id=dir2.dir3.two -a expression='False' test python.ExecTest
This will create five new tests, together with three directories:
> qmtest ls -lR
directory dir1
test python.ExecTest dir1.one
test python.ExecTest dir1.two
directory dir2
directory dir2.dir3
test python.ExecTest dir2.dir3.one
test python.ExecTest dir2.dir3.two
test python.ExecTest dir2.one
These directories are implicit suites, i.e. it is possible to only address tests within them:
> qmtest ls -lR dir2
directory dir2
directory dir2.dir3
test python.ExecTest dir2.dir3.one
test python.ExecTest dir2.dir3.two
test python.ExecTest dir2.one
The suite named "." (a single period) is the
implicit test suite corresponding to the root directory in the test
database. This suite therefore contains all tests in the database. For
example, the command
>qmtest run .
is equivalent to:
>qmtest run
Both commands run all tests in the database.
Explicit test suites are a means to refer to a set of tests for the purpose of executing them together, no matter how they are organized in the test database. For example, you may want to run a subset of all tests nightly. To do this, you create a suite, listing all tests (as well as contained suites) explicitely:
>qmtest create --id=nightly suite explicit_suite.ExplicitSuite(test_ids="['dir1.one', 'dir2.one']", suite_ids="['dir2.dir3']")>qmtest ls -l directory dir1 directory dir2 suite explicit_suite.ExplicitSuite nightly>qmtest ls -l nightly test python.ExecTest dir1.one directory dir2.dir3 test python.ExecTest dir2.one