Design Project / D /
Testing and Evaluation
Embedded Files
Di | Testing strategy
Di | Testing strategy
Recommended format
200 words / 1 page
Level Descriptors of Student Achievement
1-3 The student states a testing strategy to measure the success of the prototype.
4-6 The student Describes a testing strategy to measure the success of the prototype.
7-9 The student Justifies a testing strategy to measure the success of the prototype.
A testing strategy is essentially a plan comprising of a series of tests that will be carried out on the prototype to assess of the success of the prototype as a solution to the design problem. The testing strategy must be designed to provide data to prove the solution meets all aspects of the design specification. When devising a testing strategy, the student needs to consider:
What the test comprises
How and from/with whom the data will be generated
What type of data will be generated (qualitative or quantitative)
The relative importance of each test, providing a clear sequence for testing
Justification of how it leads to the objective determination of the success of the solution.
Testing of all aspects of the design specification should be planned in the testing strategy. However, one test may test a number of different aspects. For example:
A well-planned test based on observing a user interact with the prototype to perform a particular function may provide data on aspects related to function, ergonomics, size and weight, materials, and aspects of safety. Additional data could be gathered through a post-observation interview and taking measurements during the observation.
Qualitative data can sometimes be subjective; it can be based on opinions. Students need to consider the size of a user group to ensure there is sufficient data from which to draw objective conclusions. Also, students need to consider if it is appropriate to determine the success of the prototype against particular aspects of the design specification using qualitative data if quantitative data can provide a more objective evaluation. For example, qualitative feedback from users stating something looks weak or could be cracked easily is not appropriate to determine the durability of a product. However, carrying out performance testing (destructive), such as drop testing (dropping the product from increasing heights to see if it gets damaged) and measuring the size of any cracks in the product, provides quantitative data that can be used to inform the evaluation objectively.
The kinds of testing that students could conduct can be categorized as follows.
Performance test (quantitative* feedback): The prototype is tested under conditions specified in the design specification. Some performance testing can be destructive (for example, durability or safety testing), so the order of performance tests or the availability of more than one prototype for testing is important.
Expert appraisal (qualitative** feedback): An expert (chosen on the basis of their knowledge or experience) is asked to assess the product. This type of testing relies on the knowledge and skills of an expert in the operation of the product. Expert knowledge and advice are gained (compared to a user trial), but the expert may be biased.
User trial (qualitative feedback): The observation of people using a product and the collection of comments from people who have used a product. They are typically members of the target audience. Users may carry out tasks in different ways from those expected, and they may be inexperienced.
User research (qualitative and quantitative feedback): Obtaining users’ responses through questionnaires/surveys and interviews. This can be done individually or in focus groups.
Comparison to existing products (qualitative and quantitative feedback): A comparison against existing products can demonstrate if the prototype is as good as the competition or has better features.
Resources
Resources
...
Example testing method
Example testing method
Analyzing images of children using the bike - to measure angle at the knee determines comfort and safe use of bike (Spec 4.1). Height of 5th percentile age 3, and 95th percentile age 7 used to establish whether the angle falls within parameters of 30-60° to avoid discomfort and injury of prolonged use.
The bold yellow text is “stating” the test, the blue text is “describing” the test (the what), and the green italicized text is “justifying” the test (the why).
Dii | Evaluation of Success
Dii | Evaluation of Success
Recommended format
700 words / 2 pages
Level Descriptors of Student Achievement
1-3 The student evaluates the success of the prototype against a few aspects of the design specification with no evidence of testing.
4-6 The student evaluates the success of the prototype against some aspects of the design specification.
7-9 The student evaluates the success of the prototype against the design specification.
The prototype must be evaluated objectively against each aspect of the design specification. The objective evaluation is based on data gathered through the testing strategy. The student must have implemented the testing strategy to gather the data, and then analysed the data as evidence of the success of the prototype. Evidence of genuine testing should be included in the form of graphs and annotated photographs of performance and user testing.
Students must avoid basing their determination of the success of the solution on their own opinions; they must ensure they justify the success objectively, based on the feedback of users and clients, and the analysis of statistical data.
As students implement the testing strategy, they may think of additional aspects of the evaluation they can test. In this case, students can carry out additional tests to generate data that assists the justification of the success of the prototype, but they should be mindful to include any additional tests in the testing strategy.
Example of high performance (7–9 markband): The student evaluates the success of the prototype against the design specification.
The three aspects included below are extracted from a full evaluation of the prototype as described in the preamble to this section.
The student identifies the testing method they have implemented and demonstrates they have used a mixed approach, including personal testing, product testing with users, and specific testing that evaluates specific aspects of the design specification. The tests have allowed the student to generate data, and this is used appropriately throughout the evaluation.
The justification of the success of the prototype is supported by interviews with users, user surveys and performance testing. The student has collected data from carrying out the tests, analysed that data and presented the insights from that data. The insights are drawn either from interacting with users (feedback/observation) or through measuring aspects of performance (testing which devices would fit in different slots, and the angle at which the devices were supported for ease of use). The student does not include any personal opinion on the prototype.
The student also identifies both strengths and weaknesses of the prototype. These are supported throughout by evidence of testing. User testing not only focuses on the residents as primary users but also includes staff at the centre as secondary users. The inclusion of secondary users in the evaluation leads to further insights into the maintenance and manipulation of the prototype and makes the evaluation more complete.
Diii | Improvements
Diii | Improvements
Recommended format
250 words / 2 pages
Level Descriptors of Student Achievement
1-3 The student lists how the prototype could be improved.
4-6 The student outlines how the prototype could be improved.
7-9 The student demonstrates how the prototype could be improved, considering how individual improvements affect the design as a whole.
Once students have completed their evaluation of the prototype against the design specification, they should have a list of aspects where the prototype could be improved. The aspects for improvement need to be identified based on the results of testing, feedback from users, and the subsequent analysis and evaluation.
The improvements should be of a non-trivial nature, that is, not simply focused on how the product looks, but must address the failings of the prototype as described in the evaluation. The improvements must be based on the existing prototype and not a brand-new design—this should be considered a post-evaluation iteration of the prototype, rather than a new way to address the problem.
Students will likely address a series of improvements to different aspects of the prototype, for example, the redesign of a handle, adding parts to control movement in a mechanical system so it operates more easily, or repositioning features to make the prototype more intuitive to use. These individual improvements are likely to require the student to consider how this affects the design of the prototype as a whole. The students should consider this and present the overall changes to the prototype as a result of these improvements.
Students are expected to engage in designing to access the higher level mark band—they need to show what the improvements are: this can be demonstrated through annotated sketches or graphic, physical and CAD modelling (including rapid prototyping). Images of any modelling should be annotated clearly to indicate the changes that were made and why they were necessary. Further explanation of the changes to the design of the prototype can be provided using the 250 words of extended text.
The students could also include a presentation drawing of the whole product to highlight the improvements.
At HL, the improved prototype design becomes the starting point for HL criterion E: Detailed development of a commercial product and criterion F: Making choices for commercial production.
Examples of high performance (7–9 markband)
The student demonstrates how the prototype could be improved, considering how individual improvements affect the design as a whole. The student explores multiple aspects of the prototype that could be improved and demonstrates how these improvements would address the weaknesses identified in the evaluation.
The improvements are presented through the use of annotated sketches, CAD and annotated photographs, providing sufficient detail to understand how the improvements would affect the prototype. Further detail is provided by the inclusion of measurable aspects, for example, the 5 cm adjustable increments for the height of the table and the range of the adjustment (overall height 50 cm–100 cm), dimensions for the size and gauge of the required square tube, and the size needed to house plant pots.
Improvements to the design are addressed in multiple ways, for example, the inclusion of handles to make the hub easier to open, paired with changing the materials to make the hub lighter and, therefore, easier to move.
There is scope for further improvement by providing a presentation drawing of the whole product to highlight the improvements, that is, the next iteration of the design.
Page updated
Report abuse