Development trials. Exploratory development trials used to improve a designer's understanding of how the product works are an essential part
of the development process but have no value for validation. However, development trials that follow a protocol and are written
up in a report may save work later. When early drafts of a specification include phrases like "to be decided," feasibility
trials may show what performance levels can be expected — for example, the lifetime of a hinge, the number of operations before
cleaning is needed, and the fluid pressure at the tip of a needle.
Design reviews Formal design reviews are conducted by the design team with assistance from others who can bring a fresh view and challenge
assumptions. One meeting may be enough for a simple product but reviews for mechanical, electrical, software, and system design
can take up many days. The design review considers the FMEA, development trials, design calculations, and decisions that have
been made, all with a particularly critical eye.
Verification tests. These are the principal element of design validation. Some of these tests are performed on complete product, but others
may be done on components or sub-assemblies. This is especially relevant to long-term reliability where a pump or motor may
be tested in isolation to demonstrate that its lifetime is sufficient. The tests should include the effects of variation on
performance, including variation which comes from tolerances in manufacture and from the product's environment. The variation
can be allowed to occur naturally by using many people as test subjects, or it may be simulated by deliberate control of key
characteristics, such as the viscosity of a drug or the storage temperature.
If development tests have already established that the product's performance is satisfactory, it may be possible to write
a simple "substantial equivalence report." This justifies using the results in lieu of a verification test. A substantial
equivalence report can be used if the protocol for the verification test would be little different from the development tests
and if the design has not changed significantly.
The test protocols and pass limits are approved before the tests commence. Any deviations from the protocol must be agreed
upon, and a list of discrepancies or failures must be maintained. If the development work was thorough, there should be few
discrepancies; the purpose of the tests is confirming that the performance is within specification.
Discrepancies can be resolved with design changes, but this must be accompanied by an analysis showing how other elements
of the performance specification might be affected. In many cases, this analysis must be followed by repeating some verification
tests. If the number of discrepancies and design changes increases, it may be necessary to redefine the verification tests
as development trials and commence a new verification.
Some discrepancies can be resolved by simply amending the specification. In theory this should not happen if a top-down approach
was taken in creating the specification. In reality, part of the performance specification often is written after feasibility
studies, which were performed under ideal conditions on a laboratory bench. A test unit using production parts under "worst
case" conditions can fail if the effects of the tolerances are not considered.
A third way to resolve discrepancies is identifying assignable causes of failure — the protocol was not followed, there was
a power failure, a test lead broke. However, there is a risk that these causes become a series of excuses that are applied
until a passing result is obtained.
Validation tests. Validation tests are performed when it is not possible to objectively measure performance. They are applied, for example,
to the ease of assembly of a device, the legibility of labels, and the instruction manual. A team of about 10 people is selected
with the requirement that they are representative of the intended users and are not familiar with the product. This excludes
the design team and many engineers and managers in the quality, production, and marketing departments. The team follows the
validation protocols and their subjective assessments are recorded and compared with the pass limits. The criteria for deviations
and discrepancies that apply to verification tests also apply to validation.
Post-market surveillance. Information from real users also must be collected, but it is not part of the formal design validation for most products.
(Clinical trials are a different matter entirely.) Questionnaires and interviews provide the best feedback about ease of use
and operability and are used to confirm that the verification and validation tests were an effective surrogate for actual
use.
|