Test planning involves scheduling and estimating the system testing process, establishing process standards and describing the tests that should be carried out.
As well as helping managers allocate resources and estimate testing schedules, test plans are intended for software engineers involved in designing and carrying out system tests. They help technical staff get an overall picture of the system tests and place their own work in this context. Frewin and Hatton (Frewin and Hatton, 1986). Humphrey (Humphrey, 1989) and Kit (Kit, 1995) also include discussions on test planning.
Test planning is particularly important in large software system development. As well as setting out the testing schedule and procedures, the test plan defines the hardware and software resources that are required. This is useful for system managers who are responsible for ensuring that these resources are available to the testing team. Test plans should normally include significant amounts of contingency so that slippages in design and implementation can be accommodated and staff redeployed to other activities.
Test plans are not a static documents but evolve during the development process. Test plans change because of delays at other stages in the development process. If part of a system is incomplete, the system as a whole cannot be tested. You then have to revise the test plan to redeploy the testers to some other activity and bring them back when the software is once again available.
For small and medium-sized systems, a less formal test plan may be used, but there is still a need for a formal document to support the planning of the testing process. For some agile processes, such as extreme programming, testing is inseparable from development. Like other planning activities, test planning is also incremental. In XP, the customer is ultimately responsible for deciding how much effort should be devoted to system testing.
The structure of a test plan
Test plans obviously vary, depending on the project and the organization involved in the testing. Sections that would typically be included in a large system, are:
The testing process
A description of the major phases of the system testing process. This may be broken down into the testing of individual sub-systems, the testing of external system interfaces, etc.
Requirements traceability
Users are most interested in the system meeting its requirements and testing should be planned so that all requirements are individually tested.
Tested items
The products of the software process that are to be tested should be specified.
Testing schedule
An overall testing schedule and resource allocation. This schedule should be linked to the more general project development schedule.
Test recording procedures
It is not enough simply to run tests; the results of the tests must be systematically recorded. It must be possible to audit the testing process to check that it has been carried out correctly.
Hardware and software requirements
This section should set out the software tools required and estimated hardware utilisation.
Constraints
Constraints affecting the testing process such as staff shortages should be anticipated in this section.
System tests
This section, which may be completely separate from the test plan, defines the test cases that should be applied to the system. These tests are derived from the system requirements specification.
Testing in a plan-based software process (The V-model)
The so-called V-model of software development relates different stages of testing to activities in the software process. For each stage in the software process, there is a related testing activity. This is shown in Figure 1. The V-model is used in tightly controlled development processes, such as those used for safety-critical systems.
