Process Area 5.2 – Quality Control – SG 1 – Establish a statistically controlled test process

A statistically controlled test process is established whereby baselines that characterise the expected test process performance of the organisation’s standard test processes are established and maintained.

Introduction

The purpose of Quality Control is to statistically manage and control the test process itself. At this Level, test process performance is fully predictable and stabilised within acceptable limits.

Testing at a project level is performed using statistical methods based on representative samples in order to predict product quality and make testing more efficient.

Understanding variation is achieved, in part, by collecting and analysing measures so that (common) causes of variation can be identified and addressed to achieve predictable performance. Basically, numbers are crunched in simple or complex ways.

Specific Practices

SP 1.1 Establish test process performance objectives

This Specific Practice correlates to and builds upon the SP 1.1 Establish test measurement objectives of the Test Measurement Process Area at TMMi Level 4.

The test policy and other sources of the organisation’s performance objectives need to be studied

SP 1.2 Establish test process performance measures

Refer to the Test Measurement Process Area at TMMi Level 4 for information about selecting and establishing measures, especially SP1.2 Specify test measures.

SP 1.3 Establish test process performance baselines

There may be several performance baselines to characterise test performance for subgroups of the organisation, such as:

• application domains
• complexity
• project size
• life-cycle models.

SP 1.4 Apply statistical methods to understand the variations

Understanding variation may be achieved by collecting and analysing measures so that common causes of variation can be identified and addressed to achieve predictable performance.

All processes will show some variation in measures each time they are executed. It is useful to set upper and lower boundaries for this variation to help the test process group to separate the signals from the noise.

An important activity during the identification of causes of variation is to determine if a process variation is caused by special circumstances which may be linked to a specific time or location, or by the variation inherent in the nature of the process itself (a common or natural cause). In general it is recommended to first solve assignable causes, since these may decrease common causes.

SP 1.5 Monitor performance of the selected test processes

This is done to determine the capability to satisfy process performance objectives and to identify corrective actions.

Deployment notes

• Is it possible to see if KPIs from Level 2 have been achieved or not?
• Is there an understanding of the performance variations of the selected test processes (using the selected measures)?
• Are performance baselines available?
• Is there a history of process performance available so it is possible to see achievement over time?