Testing & Evaluation

Testing

Test Data

When we test a program, we use test data. You enter the test data as the inputs to the program. The data should try to capture a range of different scenarios - don’t just test the program with one piece of data.

There are three types of data you need to be able to use and give examples of: normal, extreme and exceptional. In this example, the program expects a number between 0 and 10 (inclusive).

Test Table

We write this data into a test table. The test table shows the data we test the program with. We put one example in each row of the table (i.e. don’t say “anything between 1 and 9” - pick one!)

In the test table, you should say what you expect the program to do. Some test tables have another column for what actually happened, and you would record this afterwards.

Types of Errors

There are three types of programming errors that you need to know about - syntax, logic and execution.

If you’re given an example of an error, you need to be able to say what type it is.

Syntax and logic errors occur when you are writing the program; execution errors happen when the program is running. Of these, syntax errors are usually the easiest to spot.

Syntax Errors

Syntax errors are quite common. You will already have experienced lots of these!

Syntax errors happen when you make a mistake in typing your code. These could be spelling mistakes, typing words in the wrong order, or using the wrong symbols.

For example:

  • priiiint instead of print

  • print(“Hello,” name) instead of print(“Hello”, name)

  • range[0, 10] when it should be range(0, 10)

Syntax errors are about breaking the rules of the language in some way.

Logic Errors

Logic errors are a bit trickier to spot. Logic errors don’t produce error messages. The program still runs, but it doesn’t do what you’d expect.


A program can translate all of its code without errors (including no run-time errors) but still not give correct results because of an error in the logic of the instructions.

For example, writing code to add two numbers instead of multiplying them, or subtracting two numbers the wrong way around.

The order of your code must be logical too; let’s look at an example…

In line 1 the variables number1 and number2 have not yet been assigned a value, so they are initially set to 0.

Regardless of the input in lines 3 and 4, the variable answer will always 0, because it was calculated as 0 in line 1. Therefore the answer outputted will always be 0.

Line 1 answer = number1 + number2

Line 2 number1 = InputBox (“Enter number 1”)

Line 3 number2 = InputBox (“Enter number 2”)

Line 4 MsgBox (“The answer is “ & answer)

Execution Errors

These are errors that are not detected by the translator but are discovered when the program is run.

A common execution error occurs when the program tries to divide by zero, generates an error and crashes.

Another example is when a program attempts to read data from a file and the file is not present in the specified path.

Execution errors are also known as run-time errors, since they are errors that prevent the instructions from being run.

Evaluation

You have to be able to evaluate a program - either yours, or one you’re given - in four categories:

  • Fitness for purpose

  • Efficient use of coding constructs

  • Robustness

  • Readability

Fitness for Purpose

If your program is fit for purpose, it means it does what it’s supposed to do. Think about how you would know what the program is supposed to do - you’d go back to the analysis!

If the program meets all of the functional requirements you set out in the analysis, then it is fit for purpose - it does all the things it’s supposed to. If it doesn’t meet one or more, it must not be fit for purpose.

When you are doing the evaluation, use the term ‘fit for purpose’ or ‘not fit for purpose’ - be explicit. Say why it is or isn’t fit for purpose, mentioning the specific requirement e.g.:

  • The program is fit for purpose, because it asks the user their age, and shows an error if they enter a negative number

  • The program is not fit for purpose, as it is only supposed to accept percentages between 0 and 100, but it accepts values outside this range

Efficiency

The word efficient means “achieving maximum productivity with minimum wasted effort or expense”.

In programming, efficiency means your program does what it’s supposed to do by using as little lines of code as possible.

This means the computer has less lines to translate and execute resulting a faster, more efficient performance.

Some easy ways to reduce code are:

  • Using loops where code repeats itself

  • Using an If Else rather than multiple IF's where possible

Robustness

A robust program doesn’t crash when the user enters exceptional data. Take the age program from earlier. If the user enters a negative number (no one has a negative age!) the program shouldn’t just crash - it should show an error message and ask them to enter it again.

You can test robustness using exceptional test data. How does the program behave if you enter exceptional data? If it crashes, the program isn’t robust. If it can cope with errors (e.g. by asking them nicely to enter it again), it is robust, and you can say this in your evaluation.

Readability

A program is readable if another programmer can look at the code and understand it.


Often teams of programmers will work on the same program code so they should help each other out and follow the rules of program readability:

  • Internal commentary to explain specific lines of code or sign and date sections of code

  • Meaningful identifiers such as variable and array names so other programmers understand their purpose based on their name

  • White space to space the code out and make it easier to read and split up into sections

  • Indentation to make loops and conditional IF constructs easier to read and understand