12.3 Program Testing and Maintenance

12.3 Lesson Resources.  The textbook covers this fairly comprehensively, so the majority of lesson presentations are not uploaded as they do not add to the knowledge out there. 

Types of Error

There are three types of error you need to be able to identify and describe:

A syntax error is an error in the grammar of the language. It should be detected by a good IDE and will prevent the program from translating (e.g. compiling) by the parser. 

A logic error is where the program fails to function as intended. These errors rarely cause the program problems (although this depends on the logic error in question), but can still have disastrous consequences to the underlying data.

A runtime error is one spotted and detected when the program is running. These can be difficult to spot as their cause may be issues outside of the program. For example, the user deletes a critical file you need to read in, etc. More typical runtime errors can be things like dividing by zero, etc.

The Watson textbook (pp 293-299) and Langfield (pp 296-308) cover the basics of testing sufficiently that there is little need for additional detail on here.  However, this page does give more detailed descriptions of the types of test you need to know.


Types of Test

Each heading is hypertext link to another website which offers more detail.

dry run

In computer programming, is a dry run is a mental run of a computer program, where the computer programmer examines the source code one step at a time and determines what it will do when run.

walkthrough

Code Walkthrough is a form of peer review in which a programmer leads the review process and the other team members ask questions and spot possible errors against development standards and other issues. The meeting is usually led by the author of the document under review and attended by other members of the team.

white-box

White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) is a method of testing software that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing). E.g. tests will be run to check all execution paths through the program, loops, etc.

black-box

Black Box Testing, also known as Behavioural Testing, is a software testing method in which the internal structure/ design/ implementation of the item being tested is not known to the tester. These tests can be functional or non-functional, though usually functional.

integration

Integration testing, also known as integration and testing (I&T), is a software development process which program units are combined and tested as groups in multiple ways. In this context, a unit is defined as the smallest testable part of an application.

alpha

Alpha testing takes place at the developer's site by the internal teams, before release to external customers. This testing is performed without the involvement of the development teams.  In alpha testing, a set of test data is usually provided and this used by the internal teat to test the software.  This ensures that testing is comprehensive and not left to chance. The program is not always completed at this stage, but is mostly working. There may be known errors or missing features.

beta

Beta testing is the final stage of testing before acceptance and before general release, and normally can involve sending the product to beta test sites outside the company for real-world exposure or offering the product for a controlled trial download over the Internet. Beta testing is often preceded by a round of testing called alpha testing.  In beta testing, the users test with their own data.


A more detailed comparison between alpha and beta testing can be found here.

acceptance

Acceptance Testing is a level of software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system's compliance with the business requirements and assess whether it is acceptable for delivery.  


The latter part is carried out by the customers of a product where a software package has been contracted out.  In the case of a mass-market product, such as a game, this is done internally. In reality, acceptance testing is usually done where a company has been paid to produce software and these tests are carried out before payment is made.


It should be emphasised that acceptance testing is not so much about finding bugs, as checking that the finished product is exactly what has been asked for in the requirements specification.  That is, has it all been programmed, every feature done and that it works as agreed.


Stub

Stub testing allows a developer to use dummy modules for testing purposes. A module can be a program or individual functions/subroutines. The value passed back is consistent and allows parts of the program to be run when they are still incomplete. 

Program Maintenance

Once the program has been released to the customer, it can go through three types of maintenance:

The book (chapter 12.3) covers these in more detail

Adaptive Maintenance

Adaptive maintenance deals with changes that are needed due to changes in the needs of the software. This could be to add new functionality, or modify the behaviour of existing code for new purposes, etc. E.g. changes in legislation, etc.

Corrective Maintenance

Corrective maintenance looks to fix errors that are found once the program has been accepted by the customer. These could be bug fixes but never new features. 

Perfective Maintenance

perfective maintenance looks to improve the performance of the program through code re-writes, etc. These do not address specific bugs, but efficiency metrics such as execution time, memory allocation, etc.