Acceptance Criteria for AMT
Estimations for the true process mean (99units) and variance (2.0%) were made from the assay control performance and from
Equation 1, respectively. For the AMT, both accuracy and precision are areas of concern because several method components
(operators, instruments, location and others likely) will change when the method is executed at the receiving laboratory.
Tests should therefore be undertaken for the overall matching of the receiving laboratory results to those of the reference
laboratory and for equivalent (intermediate) precision. Accuracy and precision limits are treated independently here for simplicity
although they both impact all cases (1A–2B).
The assay performance is at 3.0% (last n = 60)—or potentially higher if control outliers were included—when run under routine
conditions with small changes over time. AMV results yielded an intermediate precision of 2.4%. An imprecision higher than
3.0% should not be allowed because the method component is the highest contributor to the overall variance (Table 2). There
is already a relatively high likelihood (about 1.73%) of observing an out of specification (OOS) result with two observed
over the last 60 (Table 3 and Figure 1). The limit of 3.0% appears balanced between the likelihood of passing to achieve compliance
or project advancement and the likelihood for all cases 1A–2B to be continued in the future.
A recovery or matching of the expected (reference) potency of 100 plus or minus 1.5% appears reasonable. Allowing a maximum
of plus or minus 1.5% difference constitutes about one-half of the recent assay variability. The receiving laboratory should
not be allowed to test with a greater bias because further increase in the likelihood of OOSs because of the potential shifting
in the process mean compared with the target (100 units) may be seen. Overall, recoveries inside this range (98.5–101.5%)
as evidenced by the data from AMV, historical assay control, and SPC should be possible.
Acceptance Criteria for AMM
When exchanging or adding a single method component, such as a second instrument, maintaining accuracy or the matching of
historical performance should be the main considerations for reasons given above. However, accuracy and matching, and precision
could both be studied to monitor method performance characteristics. The acceptance criteria for accuracy and precision for
AMT and AMM should be derived from the product specifications with regards to assay performance (control) and process performance
(SPC data). As good estimates for all variance components may be present, acceptable criteria (at least for precision) can
be derived from the results in Table 2. Acceptance criteria for accuracy (matching) may need to be tightened to avoid a potentially
large compounding of bias from several one-directional changes e.g., 99.0–101.0% versus current system). Unless there was
no alternative, a 2.0% increase in test results should not have occurred when the reference standard was changed.
In Table 3, probability estimates are calculated based on the expected rounding of test results to specifications (90–110
units/mL). This leads to more test results falling within specifications as 89.5 is rounded up to 90 and 100.49 is rounded
down to 100. The current calculated probabilities for observing passing results (1A) are 98.64% for results above 100 units
and 99.63% for below, respectively, for a net total of 98.27%. Given a normal data distribution and the historical process
mean of 101.0 units, the probabilities for failing low results versus failing high results do currently not match. The allowed
worst-case probabilities for 1A after AMT (protocol acceptance criteria are 100 plus or minus 1.5%) are much greater towards
the positive direction and come to the total of 96.73%. Case 2A probabilities are simply the reverse of 1A (100% 21A).
If the AMT results would yield a still acceptable +1.5% bias, the predicted failure rate would have almost doubled from 1.73–3.27%.
For normal data distributions, cases 1B and 2B will be similar as results could equally differ in both directions from the
observed SPC results. The total variance predicted for measurement errors (3.8%) was calculated similar to Equation 1 from
the sum of assay control variance (3.0%) and sampling variance (2.3%). The calculation of exactly predicted probabilities
for cases 1B and 2B becomes complex and is beyond the scope of this article.
Two variance components (assay precision and sampling variance) have been identified that should be improved in light of the
1.73% predicted failure rate. This situation could easily get significantly worse after, for example, the method is transferred,
method components are exchanged, or process changes are implemented. It should now become clear why this should not be neglected,
acceptance criteria should be systematically derived as discussed above, and why regulatory guidance has recently incorporated
some of these principles.2–5