Verification and Validation: Observations for Simulation and Business Analysis

Verification and validation are complex subjects on their own (and see here and here more specifically for this discussion), and the simplest way I’ve found to describe the difference between them is that verification tests whether a solution works while validation tests whether a solution solves the problem.

Verification is the process of testing whether the system (software or otherwise) works without generating run-time errors, or at least that it’s able to handle and recover from errors. Individual operations can be verified formally by mathematical proof and informally through manual or automated testing. These can include send/receive operations, read/write operations (a subset of send/receive), calculations, branching and call operations, and user interface activities.

Validation in general is the process of testing whether the system (again, software or otherwise) is fit for its intended use, whether it meets the business need.

For simulation the idea of validation has two meanings. One is that the simulation itself accurately recreates observed behaviors for known cases. This gives confidence that it will produce accurate behaviors in novel cases. This is a subset of the test for whether the simulation then fulfills its intended use. Potential intended uses of simulations are described in detail here, but examples include design, real-time control, and training.

For business analysis the above discussion applies for computer-based systems and non-computer-based systems as normal. The BABOK (see also here), however, applies the terms not to software systems or systems in general, that business analysts work with, but to requirements. It gives the following definitions (page 134 of the 3rd ed.):

Verify Requirements: ensures that a set of requirements or designs has been developed in enough detail to be usable by a particular stakeholder, is internally consistent, and is of high quality.

Validate Requirements: ensures that a set of requirements or designs delivers business value and supports the organization’s goals and objectives.

These definitions definitely have the same character as those discussed above. They say that verification is about whether the requirements are usable in a way that will get something working, while validation is about whether the working solution provides the intended value.

I would argue that the BABOK’s wording could be more precise along these lines. The biggest problem I have is with the idea of “high quality,” which is nebulous and undefined. Quality, according to Philip B. Crosby, in his book Quality Is Free asserts that quality means meeting the established requirements. This definition implies that requirements have be be defined in a way that can be tested on an objective, pass/no-pass basis. (Subjective enterprises like art cannot be judged in this way, although its components can be.) I was introduced to this book during my first job in 1989, when the company was introducing the then-current ideas of Total Quality Management. I think these ideas have been subsumed by the ideas of Lean Six Sigma, although the Wikipedia article suggests that its ideas were more directly supplanted by ISO 9000, which a later company I worked for adopted in the late ’90s.

I propose that the BABOK definitions be reworded as follows:

Verify Requirements: ensures that a set of requirements or designs is internally consistent and has been developed in enough detail to support the implementation of a working system.

Validate Requirements: ensures that a set of requirements or designs supports the implementation of a working system that delivers business value and supports the organization’s goals and objectives.

This emphasizes that the requirements and designs need to lead to something concrete, but doesn’t demand that the implementation has been carried out before the verification and validation can be completed.

This entry was posted in Tools and methods and tagged , , , , , , . Bookmark the permalink.

Leave a Reply