Assessments as part of test process improvement

Assessments uncover the maturity of the test process and will determine if the organisation has met a specific level of maturity. The result of the assessment will help prescribe what improvements are required and an action plan for the improvements can be defined. A process improvement plan will include:

  • What needs to be done
  • When it should be started and finished (priority)
  • Who will be involved
  • Dependencies
  • Who owns the plan
  • Funding and other resources required
  • What the success criteria are

Additional assessments can be run at any time to Identify what improvements are required, or to check what improvements have been delivered. They may lead to further informal assessments or to a formal assessment.

Informal Assessments

Informal assessments have the following characteristics:

  • They are led by an experienced assessor who does not need to be formally accredited (but it is recommended that they are)
  • They can be performed by a single person
  • Corroboration of evidence is not required, so just one form of evidence is needed (for example an interview, or a document). This can significantly reduce the time it takes to perform the assessment.
  • They provide an indicative assessment of an organisation
  • They do not in themselves lead to a formal maturity rating
  • They are normally undertaken to identify improvements that need to be made or to monitor progress of a TMMi implementation
  • They can be quicker and cheaper to undertake than a formal assessment

Formal Assessments

Formal assessments have the following characteristics:

  • The must be led by an accredited lead assessor
  • They must include at least one other accredited assessor
  • Additional assessment team members do not necessarily need to be accredited (so can, for example, be assessors in training)
  • They have mandated data sources i.e.
    • interviews and
    • artefacts (for example: templates, operational documents, screen shots)
  • Interviews evidence must be corroborated by other evidence
  • They lead to an accurate and measurable maturity level rating if carried out using an accredited assessment method
  • It is necessary to calculate the number of projects to be assessed using the formula in TAMAR:
    • the greater of 2 or [sum of (domains, development methods & project sizes) + Log 10 (number of projects)]

Assessment Criteria Quick Comparison

Type Assessment Team Lead Team Size Evidence Capability Rating
InformalExperienced assessorAt least oneOne type of evidence required e.g. interviewNo rating against TMMi is produced.Used for a quick check assessment to gain a rough understanding to various areas maturity and improvement opportunities
FormalAccredited Lead AssessorAt least two – an accredited Lead Assessor and at least One other Accredited AssessorStaff interviews and document study required.Other types of evidence, such as questionnaires, are recommendedVerifiable benchmark rating of the organisation against TMMi is produced.Strengths and weaknesses can be identified in detail, including full gap analysis