The T&E Report outlines how you will verify that the system or some element of the system meets the requirements outlined in your A-Spec. It ensures that the appropriate aspects of the system are evaluated for compliance with the specified requirements.
The T&E report is an interim artifact during the execution of the Concept Definition life cycle phase (Kossiakoff SE P&P Chapters 7, 17).
Include a detailed test plan that covers what tests will be performed, how they will be conducted, and the criteria for success. The plan should also identify any required test equipment, environments, and resources.
Outline the methods you will use to verify each requirement. The four primary verification methods are
Test
Demonstration
Analysis
Inspection
Each requirement in your A-Spec should be linked to at least one of these methods.
Maintain strong traceability between the requirements in your A-Spec and the corresponding test cases in the T&E Report. This ensures that every requirement is verified and that nothing is overlooked.
Develop detailed test cases that describe the specific procedures for testing each requirement. These should include the test objectives, test setup, steps to execute the test, and expected results.
Lack of Detail: A common issue is the lack of detail in test cases, which can lead to ambiguity and inconsistent test results. Make sure each test case is thorough and leaves no room for misinterpretation.
Inadequate Coverage: Ensure that the appropriate requirements from your A-Spec are covered in the T&E Report. Missing requirements can lead to gaps in verification and potential system failures.
Overlooking Non-Functional Requirements: While functional requirements often receive the most attention, non-functional requirements such as reliability, maintainability, and usability are equally important and should be included in your test plan.
Define clear metrics and thresholds for success. These should be directly linked to the requirements and should be objective, measurable, and repeatable.
Establish clear pass/fail criteria for each test. This helps in determining whether the system meets the required standards or if further development and testing are needed.
Consider the need for multiple phases of testing, such as component testing, integration testing, and system testing. Each phase should build on the previous one, gradually increasing the scope and complexity of the tests.
Plan for regression testing to ensure that changes or fixes do not introduce new issues. This is especially important if your project undergoes significant modifications during development.
Select a portion of your system’s capability (can be (and usually is) centered on a subsystem or component).
List your test objectives.
Trace the relevant requirements to your test objectives.
Describe the test environment (facility, operators, location, etc.)
Test equipment needed.
Test subjects and articles that will participate.
Desired results for the test to be successful.
Metrics to be collected.
General description of the test execution (what are the high level steps?)
T&E Checklist Credit: Steve Biemer