04.05 - Squash-TA Reporting

Configuration

The configuration of the reports is done in the <exporters> sections of your project pom.xml:

One exporter tag per reports you want to activate.
implementation attribute is mandatory and is specific to each report
Child tag ( of the exporter tag ) are specific to each report

Sample configuration

...

</exporter>

</exporters>

...

</configuration>

Squash-TA HTML report

Since Squash-TA framework v1.6.0, a HTML report is available.

HTML report configuration

To activate the HTML report in your project use "org.squashtest.ta.commons.exporter.html.HtmlSuiteResultExporter" as value for the implementation attribute.

HTML report configuration sample

...

</exporters>

...

</configuration>

@since Squash-TA framework 1.7.0 there is a simpler solution to activate the html report:

...

<html/>

</exporters>

...

</configuration>

Note :

- The two possibilities to activate the html report are equivalent
- The first solution still working with Squash-TA framework 1.7.0 and above

HTML report

At the end of the execution (run or test-list) the HTML report (squash-ta-report.html) is available, by default, in the directory: 'my.squash.ta.project/target/SquashTA/html-reports'. Here are some screen capture of this report ( For a full sample see attach file: Execution report - execution-report-test-sample.pdf )

Test suite summary:

Ecosystem summary:

Target Initialisation summary: (@since Squash-TA framework 1.7.0)

Test script result:

Squash-TA Surefire report

Surefire report Configuration

To activate the surefire report in your project use "org.squashtest.ta.commons.exporter.surefire.SurefireSuiteResultExporter" as value for the implementation attribute. Two properties are available for this exporter:

- exportAttached: When set to true, the attachment produce by the instructions are generate in the surefire-reports directory.
- jenkinsAttachmentMode: When set to true, the attachment directory tree is modified. This modified directory tree is useful in order to make the Jenkins JUnit attachment Plugin work properly. (Squash-TA server).

Surefire reprot configuration sample

...

</exporter>

</exporters>

...

</configuration>

@since Squash-TA framework 1.7.0 there is a simpler solution to activate the surefire report:

Zone de texte

...

</surrefire>

</exporters>

...

</configuration>

Note :

- The two possibilities to activate the surefire report are equivalent
- The first solution still working with Squash-TA framework 1.7.0 and above
- @Since Squash-TA framework 1.7.0, the exportAttached parameter is set to true by default, whereas it was set to false before.

Surefire report

Execution reports are created in the 'my.squash.ta.project/target/SquashTA/surefire-reports'. The target directory is created during the execution.

The test cases execution with the used maven configuration provides surefire reports. There is one surefire report per ecosystem. Since Squash-TA framework v1.6.0 more details are given concerning the executed instructions. ( See full example in attach file: TEST-tests.F01.ScriptsTA.Set4.xml)

Example of a surefire report:

<?xml version="1.0" encoding="UTF-8"?>

Summary

[SUCCESS] LOAD file.txt AS file

[ERROR] VERIFY wrongId DOES contain WITH $(Hello)

[SUCCESS] $(Hello) --> {{__temp792}}

[ERROR] VERIFY wrongId DOES contain WITH {{__temp792}}

org.squashtest.ta.backbone.exception.ResourceNotFoundException: VERIFY wrongId DOES contain WITH {{__temp792}}: Cannot apply assertion : . SCOPE_ECOSYSTEM:wrongId does not exist in this test context: you must load it first.

at org.squashtest.ta.backbone.engine.instructionrunner.AbstractDefaultInstructionRunner.throwResourceNotFound(AbstractDefaultInstructionRunner.java:146)

at org.squashtest.ta.backbone.engine.instructionrunner.AbstractDefaultInstructionRunner.throwResourceNotFound(AbstractDefaultInstructionRunner.java:142)

at org.squashtest.ta.backbone.engine.instructionrunner.AbstractDefaultInstructionRunner.fetchResourceOrFail(AbstractDefaultInstructionRunner.java:105)

at org.squashtest.ta.backbone.engine.instructionrunner.DefaultBinaryAssertionRunner.getActualResult(DefaultBinaryAssertionRunner.java:136)

at org.squashtest.ta.backbone.engine.instructionrunner.DefaultBinaryAssertionRunner.doRun(DefaultBinaryAssertionRunner.java:85)

...

A surefire report is an 'xml' file which contains the results of the ecosystem TA-scripts execution. The first result of the listing is always the result of the ecosystem setup (setup.ta) even if the file doesn't exist. The last result of the listing is always the result of the ecosystem teardown (teardown.ta) even if the file doesn't exist.

Three kind of results are possible:

The test passed: in this case the surefire report provides only the name and the execution time of the test case.

- The test failed (because of a false assertion): In this case the surefire report provides the name and the execution time of the test case and indicates that a 'failure' occures and give the associated trace.

Summary

SETUP :

[SUCCESS] LOAD file.txt AS file

[FAIL] ASSERT file DOES contain WITH $(rubbish)

[SUCCESS] $(rubbish) --> {{__temp306}}

[FAIL] ASSERT file DOES contain WITH {{__temp306}}

Trace here

[NOT_RUN] ASSERT file DOES contain WITH $(Hello)

[NOT_RUN] $(Hello) --> {{__temp152}}

[NOT_RUN] ASSERT file DOES contain WITH {{__temp152}}

TEST :

[NOT_RUN] ASSERT file DOES contain WITH $(Hello)

[NOT_RUN] $(Hello) --> {{__temp597}}

[NOT_RUN] ASSERT file DOES contain WITH {{__temp597}}

TEARDOWN :

[SUCCESS] ASSERT file DOES contain WITH $(Hello)

[SUCCESS] $(Hello) --> {{__temp529}}

[SUCCESS] ASSERT file DOES contain WITH {{__temp529}}

</failure>

</testcase>

- The test execution failed: In this case the surefire report provides the name and the execution time of the test case and indicates that an 'error' occurs and give the associated trace.

Summary

SETUP :

[SUCCESS] LOAD file.txt AS file

[ERROR] ASSERT wrongId DOES contain WITH $(Hello)

[SUCCESS] $(Hello) --> {{__temp238}}

[ERROR] ASSERT wrongId DOES contain WITH {{__temp238}}

Trace here

[NOT_RUN] ASSERT file DOES contain WITH $(Hello)

[NOT_RUN] $(Hello) --> {{__temp362}}

[NOT_RUN] ASSERT file DOES contain WITH {{__temp362}}

TEST :

[NOT_RUN] ASSERT file DOES contain WITH $(Hello)

[NOT_RUN] $(Hello) --> {{__temp345}}

[NOT_RUN] ASSERT file DOES contain WITH {{__temp345}}

TEARDOWN :

[SUCCESS] ASSERT file DOES contain WITH $(Hello)

[SUCCESS] $(Hello) --> {{__temp875}}

[SUCCESS] ASSERT file DOES contain WITH {{__temp875}}

</error>

</testcase>

Inside the target directory, there are also attached documents produced by TA-scripts to make easier diagnosis when an error occurs (snapshots, diff-reports, detailed logged...).

Squash-TA HTML Summary

Since Squash TA 1.10, users may change the detail level of the HTML report.

The HTML Summary is a less detailed version of the HTML report.

You can replace the balise of the HTML report (<html> [...] </html>) by the light version this way :

Zone de texte

<outputDirectoryName>directoryOfHtmlSummary</outputDirectoryName>

</htmlSummary>

</exporters>

You can lighten the TOC if you add the next tag:

Zone de texte

<includeHeader>false</includeHeader>

It's possible to have both versions together. You must put the two tag in the tag <exporters>...</exporters>

Example

<outputDirectoryName>directoryOfHtmlSummary</outputDirectoryName>

<includeHeader>false</includeHeader>

</htmlSummary>

<html/>

</exporters>

Page updated

Report abuse