Testing Guidelines

Introduction

It is important that the testing you carry out on your solution shows that your solution both works and is robust.

You may choose to undertake the testing as you develop the program, during or at the end of each iteration. Equally, testing can be undertaken once the program is completely finished.

Regardless of when you complete the tests, you should make sure to include evidence of testing in the Testing section of your report, or at least to other previous pages that have evidence of tests. There is no need to repeat tests that you have carried out earlier.

Any of the following may be used for evidence of testing:

• Format test tables showing inputs and outputs.

• Video evidence of the program working with different inputs/scenarios.

• Screenshots with annotation of parts of the program working.

• References to other sections that show testing during iterative development.

Sufficient Testing

Be aware that the testing section of your report is only worth just over 10% of the total marks, but some of the evidence that you provide of testing could demonstrate evidence for your Technical Solution.

When you carry out the tests it is essential that you focus on the core requirements for the solution. If you are providing evidence of parts of the program that are not requirements for the solution keep this to a minimum as it is not required.

When testing it is important that you select representative samples of tests and test data that will demonstrate the key parts of your program working.

Iterative Testing

Testing iteratively can be done at anytime at any point during an iteration and does not always have to be after you have completed the code.

Remember that when you compile and run the program you are already carrying out testing. At this stage you should record what has been tested and check appropriate tests at this stage of the development to make sure all scenarios are checked.

Failed tests show solid development strategy and understanding. No one writes perfect code for a complex piece of software on their first try. The exam board expect there to be errors that you will need to correct during the time completing your project.

During iterative development there is no need to repeatedly test the same thing unless you think that a change somewhere else in the code could effect the test results.

Strong Iterative Test Features

Strong iterative testing will include the following:

• Summarise the tests that worked using a range of suitable evidence.

• Demonstrate a few of the tests which have passed, focusing on the more interesting ones or the ones that show the system has met a core objective.

• Discuss the reason why non-trivial tests failed, showing a clear line of thought to a solution.

• Prove the error has been fixed.

Post-Development Testing

Once you have completed your program you will need to make sure that it is fully tested. This testing will look at the system as a whole.

Remember that most of the program such as modules, classes and algorithms may have already been tested as part of the iterative testing. The focus of post-development testing is to show that everything works together as one robust program.

The tests that are undertaken at this time will probably be similar to how and end user would use the system. You will need to think about the user would use the system and mimic this use.

You may want to copy the main objectives into the the section and then design suitable tests to check that these objectives have been met.

Strong Post-Development Testing Features

Strong post-development testing will include the following:

• Avoid repeating any of the iterative tests that have already been carried out.

• Focus on the system as a whole, and not individual parts (unless the core requirements require this part to be tested and it has not previously been tested).

• Mimic real-life interactions.

• Directly link to the original requirements in the objectives in the design section.

Evidencing Testing

You should look at employing a a mixture of the following to evidence your testing.

Test Tables

Tables can be a very good way of recording test evidence. These can easily be based on the objectives and are a methodical way of showing both the robustness and completeness of your project.

Comments can be added to tests where they demonstrate a line of thought when tests fail and they will show that you are considering what is causing the error.

Video

It is advised to use a video to evidence your solution because this can quickly demonstrate that your project is robust and that it works. Video enables you to demonstrate something quickly and easily that would require a number of screenshots.

Video also enables you to easily show movement in games or transitions through a complex system. It can also record sound which means that you are able to describe what you are demonstrating so that the viewer can gauge its relevance to the projects objectives.

The length of the video produced needs to be long enough to demonstrate that the system is fully working, but you need to avoid endless trivial tests or making your teacher and moderator have to sit through a video that is unnecessarily long. Five to ten minutes should be adequate to show that your project meets the objectives set and also demonstrate the program is robust. If your solution requires a longer video then you should use more time.

Make sure you plan the tests that you will need to demonstrate beforehand so that you can be concise and cover all the important tests you want to show. If you are going to speak to explain what you are demonstrating you may find it easier to prepare a script or the voice over could be added later in editing software.

If you are providing evidence in a video you will still need to complete test tables, but these will refer to a time in the video.

Benefits

• Shows transition and movement

• Very suited to games

• Can save a lot of time over screenshot evidence

• Screen recording software can be free

Drawbacks

• Can generate large files

• Harder to reference

• Must timestamp every reference

• Not 'in the document' so must be clear where you are using the video as evidence

Screenshots

Screenshots are the easiest method of evidence recording. They can be created without any need for additional software and they allow you to evidence parts of your program working and demonstrate that data and instructions inputted has the desired effect.

Remember that you could also use the snipping tool on a Windows computer to select areas of the screen you want to capture.

There are drawbacks to screenshots. The most overlooked one is that your document file size will increase significantly. Screenshots can also cause a number of issues with formatting in lots of word processing applications, so that you will need to be careful that when you edit the document you check the images are still in the right place.

You will also need to make sure that any screenshots that you use are clear and easy to read by your teacher and the moderator. Check that the resolution of the image makes it clear and if there is any text that you want to be able to be read it needs to be big enough to read.

Benefits

• Quick to make

• Do not require special software

• Good at capturing development over time

Drawbacks

• Usually needs cropping

• You will need to consider word wrapping and formatting

• Potential for poor resolution

• Shrinking screenshots can leave them unreadable

• Taking too many screenshots will slow you down

• Increases file size of document

Photos

Taking photos with a smart phone or digital camera can also be an option for evidencing you solution. This option will probably be more appropriate if you are working with physical computing solutions. For example, a photograph of the components put together to form the hardware for a project would demonstrate how the hardware has been put together.

Photos will need to be cropped and transferred to the report for them to be included.

Benefits

• Easy to use

• Do not require special software

• Can get good quality from smartphones

• Good a capturing physical computing development over time

• Generally easy to transfer

Drawbacks

• Usually need cropping/editing

• File sizes from smartphones tend to be large.

• You will need to consider word wrapping and formatting

• Shrinking photos of screens can leave text unreadable

• makes document file sizes large