For cases where the ANOVA ρ-value is less than 0.05, secondary acceptance criteria should be established for the comparison-of-means and variability
of the results to demonstrate the overall lab-to-lab reproducibility of test results. It is advisable to include a numerical
fall-back limit (or percentage) because the likelihood of observing statistical differences may increase with the precision
of the test method. In addition, some differences (bias) between instruments, operator performances and days are expected.9 We should tailor our acceptance criteria for overall (intermediate) precision and for the maximum tolerated difference between
mean laboratory results (accuracy or matching) to minimize the likelihood of obtaining OOS results (2A and 2B) or 1B results.9 The setting and justification of all acceptance criteria must strike a balance and is a critical part of each protocol.
A detailed AMT case study was presented elsewhere.9
ANALYTICAL METHOD COMPARABILITY
Table 2. General guidance for using appropriate statistics to assess method comparability from validation characteristics
per ICH Q2A/B.
|
As we need to demonstrate equality or improvement whenever approved methods are replaced, which and how are method performance
characteristics compared? Table 2 provides guidance on which validation characteristics to use for comparability protocols
for each assay type per ICH Q2A/B. All qualitative tests should contain a comparison of hit-to-miss ratios (for "specificity")
between the approved method and the new method. If a qualitative limit test is exchanged, the detection limit (DL) of the
new method should be compared and should be equal or lower for the new method. For all quantitative methods, the method performance
characteristics accuracy and precision (intermediate precision) should be compared.13
It is of great regulatory concern whether results may change overall by drifting (change in "accuracy" or "matching") or by
an increase in data spreading ("intermediate precision"). An increase in data spreading or lack of precision will mostly increase
the likelihood of observing cases 1B or 2B and should be avoided. A drift in results or "lack of matching" may require a change
in the specifications. This drift in release results can occur in two directions, lower or higher results. Both directions
are not acceptable outcomes for the demonstration of accuracy, and testing for equivalence between methods is therefore correct.13 The goal of comparisons of other characteristics (e.g., DL) is different such that two outcomes (equal or better) are acceptable.
For example, for the comparison of DLs, the two outcomes of having either an equal or lower DL would both be acceptable.13 A third comparison category is the demonstration of noninferiority and is usually the easiest to pass for comparability.
However, we should keep in mind that the use of any of the comparability categories (noninferiority, equivalence, superiority)
using ICH E9 and Committee for Proprietary Medicinal Products (CPMP) guidance documents should be properly chosen and justified.13,17–19 In other words, noninferiority testing may be justified for the comparison of some primary characteristic such as DL if
other secondary criteria (e.g., increased number of tests or test samples) can compensate for the small level of inferiority
of the primary comparison characteristic.13
Quantitative limits (QLs) could also be compared. However, both QLs would have to be estimated by the same principle (e.g.,
estimated by regression analysis). A low QL is desirable as it will let us quantitatively report and monitor low-value results
by SPC. There are several ways to compare QLs. For example, we could compare the regression coefficients of both linear assay
response curves to estimate both QLs and would also get a general idea how accuracy and precision characteristics compare
over the assay range. Table 2 constitutes a general guidance. Particular examples for noninferiority, equivalence, and superiority
testing to demonstrate method comparability were provided and discussed elsewhere.13
|
