Prototype, verification and validation: what evidence is needed at each stage 

The terms verification and validation are used interchangeably in many engineering programmes, which creates problems that are usually discovered at the wrong moment. A programme that completes verification and believes it has validated its product has answered a different question from the one it needed to answer. Understanding the distinction, and what evidence is required at each stage, is foundational to managing a development programme without expensive late surprises. 

The cost of getting verification and validation sequence wrong is not theoretical. In Rolls-Royce work presented through INCOSE, requirements validation identified 36% of issues at the lowest cost factor, while issues first found at bench or test-rig stage carried a cost factor of 50 and those first found in service carried a cost factor of 200. That is why programmes need to be explicit about what evidence is required at each stage and what question that evidence is meant to answer.

What is the difference between verification and validation in engineering? 

Verification is the process of demonstrating that a design meets its specified requirements. It is, in principle, a tractable problem: if the requirements are defined specifically enough to be measurable, and the design can be assessed against those measures, the verification question can be answered. 

In practice, verification is complicated by two things. The first is that requirements are often not defined specifically enough to be measurable, in which case verification becomes a matter of interpretation rather than demonstration. The second is that verification is typically conducted on prototypes, which may not be fully representative of production intent, so the extent to which prototype verification evidence transfers to the production design needs to be assessed explicitly. 

Verification methods include analysis, inspection, demonstration and test. The appropriate method depends on the nature of the requirement. A structural requirement might be verified by FEA corroborated by physical test. A dimensional requirement might be verified by inspection alone. A performance requirement will usually require physical test under defined conditions. The verification plan should specify the method, the acceptance criterion and the conditions under which each requirement will be assessed.

What validation tests that verification does not

Validation is a different question. Where verification asks whether the design meets the specification, validation asks whether the product, when used as intended by its intended users in its intended environment, does what it is supposed to do. The specification is a proxy for intended use. Validation tests whether the proxy was adequate. 

This distinction matters because specifications are written by people who are trying to anticipate use, and that anticipation is always imperfect. Edge cases that were not considered in the specification, operating conditions that were not fully characterised, user behaviours that were not predicted: these are the things that validation exposes. A product that passes all its verification tests can still fail in service if the specification did not adequately capture intended use. 

Validation therefore requires a clear account of who the product is for, how they will use it, in what environment, and over what timeframe. Without that account, validation activities cannot be designed with confidence that they are testing the things that matter. 

What evidence is needed at prototype stage — and what it cannot prove

At prototype stage, the primary questions are whether the concept is physically realisable and whether the key critical issues identified at feasibility can be resolved. The prototype should be designed to answer specific questions, and the validation plan should specify what a satisfactory answer looks like. 

Evidence at prototype stage is inherently provisional. Prototypes are rarely fully representative of production intent in terms of materials, processes, tolerances or assembly. The evidence they produce is most useful for confirming or disproving specific technical hypotheses, not for making broad claims about product performance. 

The risk at prototype stage is over-interpretation: treating a prototype test result as more broadly applicable than it is. A prototype that passes a performance test under controlled conditions has shown that the concept can achieve that performance under those conditions. It has not shown that a production unit will achieve it under real-world conditions. The gap between those two claims needs to be managed explicitly in the development plan. 

What evidence a funding or development gate actually requires

Funding and development gates require evidence that is calibrated to the decision being made. An investor committing to a Series A needs different evidence from a programme board deciding whether to release tooling budget. In both cases, the evidence should be sufficient to allow the decision to be made with a clear understanding of what remains unresolved. 

A common failure at development gates is presenting evidence that demonstrates activity rather than resolves questions. Test results that show a system operating, without specifying what was being tested, against what criterion, and with what outcome, are not evidence in the useful sense. They are documentation of effort. 

Useful evidence at a development gate includes: a clear statement of what the critical issues were at the last gate, what work was done to resolve each issue, what the results showed, and what remains uncertain going forward. That structure allows a decision-maker to assess the programme’s technical trajectory rather than its current status in isolation.

The relationship between verification and validation: what order they must follow

Verification and validation are related but sequential. Verification should be substantially complete before validation begins, because validation tests the product against its intended use, and if the product does not meet its specification the validation results are difficult to interpret. 

In practice, many programmes run verification and validation activities in parallel, which is reasonable if managed carefully. What should not happen is for validation evidence to be used to substitute for verification, or for verification to be presented as validation. Each has a specific role in the evidence base, and conflating them reduces the quality of both. 

How to set the right evidence standard at each programme stage

The appropriate evidence standard at each stage of a development programme is a function of what decisions need to be made and what the consequences of a wrong decision are. A programme developing safety-critical machinery needs a higher evidence standard than one developing a non-critical industrial tool. A programme approaching regulatory submission needs evidence that meets the regulator’s standard, not just the programme team’s internal standard of confidence. 

Getting the evidence standard right at each stage means neither collecting too little evidence, which leaves the programme exposed to late-stage discoveries, nor collecting too much, which wastes time and budget on evidence that does not materially improve the quality of the decisions being made.

Frequently asked questions

What is the difference between verification and validation in engineering? 

Verification asks whether the design meets the specification. Validation asks whether the product, as used by its intended users in its intended environment, does what it is supposed to do. A product can pass all its verification tests and still fail in service if the specification did not adequately capture intended use. Both are necessary; neither substitutes for the other. 

What evidence is needed at a prototype gate for an industrial machine?

At a prototype gate for an industrial machine, the evidence should demonstrate: that the concept is physically realisable and the key critical issues from the feasibility stage have been addressed; that the prototype has been tested against specific, pre-agreed acceptance criteria rather than just operated; and that what remains unresolved is clearly identified, with a plan for resolving it in the next phase. Evidence that demonstrates activity rather than resolves questions does not serve the gate decision. 

How do you structure a validation plan for an engineered product programme?

A validation plan for an engineered product programme should begin with a clear account of intended use: who operates the product, in what environment, at what duty cycle, and over what service life. From that account, the plan should define what needs to be demonstrated, by what method, under what conditions, and to what acceptance criterion. The plan should distinguish validation activities from verification activities and specify the evidence standard appropriate to each programme gate.  

Can verification evidence from a prototype be used to validate the production design? 

Not directly. Prototype verification evidence is specific to the prototype configuration, which may differ from the production design in materials, tolerances, assembly methods or manufacturing processes. The extent to which prototype evidence transfers to the production design needs to be assessed explicitly in the verification and validation plan. Where the differences are significant, additional verification or validation activities on production-representative hardware may be required.

Related resources

Precision Control Systems

Case Study: Dynamic bearing test rig