If the AMT results would yield a still acceptable +1.5% bias, the predicted failure rate would have almost doubled from 1.73–3.27%.
For normal data distributions, cases 1B and 2B will always be similar as results could equally differ in both directions from
the observed SPC results. The total variance predicted for measurement errors (3.8%) was calculated similar to Equation 1
from the sum of assay control variance (3.0%) and sampling variance (2.3%). The calculation of exactly predicted probabilities
for cases 1B and 2B becomes complex and is beyond the scope of this article.
Two variance components (assay precision and sampling variance) have been identified that should be improved in light of the
1.73% predicted failure rate. This situation could easily get significantly worse after, for example, the method is transferred,
method components are exchanged, or process changes are implemented. It should now become clear why this should not be neglected
and acceptance criteria should be systematically derived as discussed above, and why regulatory guidance (PAT and risk-based
validation) has recently incorporated some of these principles.3–6
Improving the situation
It is usually more cumbersome to implement variance (precision) improvements versus matching (accuracy) improvements. Precision
improvements usually mean more samples, more testing and more tightly controlled processes, and are therefore costly changes.
Accuracy or bias in testing is usually easier to affect by simply switching to another instrument, reference standard, etc.
However, the pushing of results into only one direction can backfire and make the situation much worse later when small changes
can add up and are not visible by themselves (because of poor precision). Decreasing the overall measurement errors or uncertainty
in test results by increasing precision has a better long-term affect because any bias in results (from production, sampling
or method changes) will be more visible and will appear sharper. Following the potency bioassay example, the sampling and
testing scheme is briefly illustrated below.
Potency in-process sampling and testing scheme
To decrease the relatively high variance in the observed potency results (SPC), a variance analysis is needed. The sampling
and assay set up for the downstream in-process potency test would then look like this (n = number of samples):
- collect n = 1 in-process sample
- split sample into two aliquots
- run two independent assay runs, each generating n = 3 results (total of n = 6 results).8
In the example, reproducibility could be significantly improved by collecting n = 3 independent samples, each run in 3 independent
assays with each having 3 replicates (total of n = 9 results). Because of the current assay set-up and sampling process, increasing
the number of samples from n = 1 to n = 3 should significantly improve precision by a factor of 1.73 (square-root of 3, see
Equation 2).8
To a lesser extent, but likely still worthwhile, an increase in assay runs from two to three runs should further improve the
situation. Another more easily implemented improvement would be to run the assay control at only ±2 s.d. (instead of 3 s.d.).
This test system suitability change should also significantly improve the overall SPC variance (Figure 1). This improvement
will lead to an expected 4% higher rate for invalid assay runs. This will be a relatively small price to pay when considered
the predicted return.
Stephan O. Krause, PhD, is the manager of QC Technical Services and Compendial Liaison at Bayer Healthcare Pharmaceuticals, 800 Dwight Way, Box 1986, Berkeley, CA 94701, tel. 510.705.4191, stephan.krause.b@bayer.com
|
