Maturity Level 5 – Optimisation

The achievement of previous test process improvement goals at Levels 2 through 4 has created an infrastructure for testing that supports a defined and measured process. At TMMi Maturity Level 5, an organisation is capable of continually improving its processes based on a quantitative understanding of statistically controlled processes.

Improving test process performance is carried out through incremental and innovative process and technological improvements. The testing methods and techniques are optimised and there is a continuous focus on fine-tuning and process improvement.

To support the continuous improvement of the test process infrastructure, a permanent test process improvement group, often called a test process group (TPG). Support for a TPG begins at TMMi Level 3 (Test Organisation). At TMMi Levels 4 and 5, more high-level practices are introduced, such as identifying reusable test assets and developing and maintaining the test asset library.

An established Defect Prevention Process Area identifies and analyses common causes of defects across the development life cycle and defines actions to prevent similar defects from occurring in the future. Outliers to test process performance, as identified as part of process quality control, are analysed to address their causes as part of Defect Prevention.

The test process is now statistically managed by means of the Process Area Quality Control . Statistical sampling, measurements of confidence levels, trustworthiness and reliability drive the test process. The test process is characterised by sampling-based quality measurements.

The Process Area Test Process Optimisation introduces mechanisms to fine-tune and continuously improve testing. There is an established procedure to identify process enhancements as well as to select and evaluate new testing technologies. Tools support the test process (as much as they are effective) during test design, test execution, regression testing, test case management, defect collection and analysis and so on. Process and test-ware reuse across the organisation is also common practice and is supported by a test asset library.

There are 3 Process Areas in this Level.

  • PA 5.1 Defect Prevention
    • Testing is a process with the objective to prevent defects.
    • Processes are in place to identify and analyse common causes of defects across the development life cycle
    • Actions are taken to prevent similar defects from occurring in the future.
  • PA 5.2 Quality Control
    • Testing is now a completely defined process and it is possible to control the costs and the testing effectiveness.
    • Testing at a project level is performed using statistical methods based on representative samples in order to predict product quality and make testing more efficient.
    • Test process performance is fully predictable and stabilised with acceptable limits
  • PA 5.3 Test Process Optimisation
  • At TMMi Maturity Level 5, an organisation continually improves its processes based on a quantitative understanding of the common causes of variation inherent in processes.
    • Improving test process performance is carried out through incremental and innovative process and technological improvements.
    • The methods and techniques are optimised and there is a continuous focus on fine-tuning and test process improvement.

The three Process Areas in this Level are highly interrelated:

  • Defect Prevention supports Quality Control, by analysing outliers to process performance and by implementing practices for defect causal analysis and prevention of defect re-occurrence.
  • Quality Control contributes to Test Process Optimisation, and
  • Test Process Optimisation supports both Defect Prevention and Quality Control, for example by implementing the test improvement proposals.

All of these Process Areas are, in turn, supported by the practices that were acquired when implementing the lower-level Process Areas.