When you execute your test through the 'run' goal, you define the test suite to execute ( aka the list of tests to execute ) through the "ta.test.suite" parameter (More details on 'run' goal here). You could define your test suite by providing to this parameter filters (for filtered execution) or data structured in json ( for filtered or ordedered execution)
"ta.test.suite": filters (filtered execution)
The ta.test.suite parameter could be a list of filters separated by comma. A filter could be:
- The test case path, absolute or relative to the automation project "tests" directory
- A path using character joker which select matching test script inside "tests" directory
- for any directory
- * for 0,1 or many characters
- A regular expression, using the format regex'<myRegex>', which select matching test script inside "tests" directory
In the sample below, ta.test.suite is compound of two filters:
- "foo/bar/baz.ta" : will select file foo/bar/baz.ta in "tests" directory (if it exists) for execution
- "sample/**/*.ta : will select for execution all file in "tests/sample" directory and its subdirectoy which name finish by ".ta"
"ta.test.suite": json data (@Since Squash-TA framework 1.8.0)
Through json data you could do a filtered execution or an ordered execution.
In the json data you could provide filters, as defined in the previous section, by using the syntax below:
You could provide, in addition, some global parameters (More details on global parameters here TODO add link) :
The other possibility, in json format, is to provide the list of tests to execute:
Where for each test:
- "script" is the the path to the test to execute relative to the "tests" directory. This property is mandatory.
- "id" is the test execution identifier. Only useful for Squash TM - Squash TA link. However, if "id" is define for one test then it should be define for all tests of the test suite.
- "param" is a list of parameters (key/value) associate to the test. This property is optional.
As for json filtered execution, global parameters are optionals
When no "param" and "id" properties are defined for test, it's possible to use a simpler syntax:
Json data could be provided through a string or a file.
Json provided through a String
Note that the double quote surrounding properties and value of the json data has been replaced. You could:
- replaced them by by simple quote (as it's done in the example)
- escaping the double quote \"
Json provided through a file
Where pathToMyJsonFile is the path to the json data file. This path could absolute or relative to the root directory of the automation project.
Filtered execution vs Ordered execution
When you doing a filtered execution you provide filters. The list of test to execute is compound of all the tests in "tests" directory whose path match the filter. With this kind of execution:
- A test could be execute only once during an execution
- There is no execution order.
- You can't provide specific parameters to the script (However you can provide global parameters through json data)
When you do a filtered execution you provide the list of tests to execute through json format. With this kind of execution:
- A test could be execute as many time as needed
- The tests are executed in the order that they were declared in the json data if the tests are in the same ecosystem. If the tests are not in the same ecosystem they are executed ecosystem by ecosystem. That means we execute all the tests of the first ecosystem used ( in the order they are declared) then the tests of the second ecosytem are executed, etc
- Parameters could be specified for each test