Current Regulatory Guidance Documents
Unfortunately, there is currently no clear guidance available on how to demonstrate the equivalence of the candidate method.
Several guidance documents contain general instructions that are helpful but do not provide sufficient detail on which characteristic
to compare and how to compare them. Also, the scope and intention of some guidelines were not designed for the purpose of
comparing a candidate method's performance to the current one. However, the following documents provide some guidance for
method comparison and/or use of statistical tests:
ISPE, Good Practice Guide on Technology Transfer, 2003.
ICH E9, Statistical Principles for Clinical Trials, London, March 1998.
CPMP, Points to Consider on Switching Between Superiority and Non-Inferiority, London, 27 July 2000.
CPMP, Points to Consider on the Choice of Non-Inferiority Margin, London, 26 February 2004)
PDA's Technical Report No. 33., Evaluation, Validation and Implementation of New Microbiological Testing Methods.
ASTM International's D4855-97, Standard Practice for Comparing Methods.
All of the guidance documents listed can be used to establish a firm's standard procedure for method comparability studies.
These and possibly other general guidelines constitute good base references if followed as written, and provide a good regulatory
Demonstrating Non-Inferiority, Equivalence, and Superiority
Of all guidance documents described, the ICH E9 and CPMP guidelines provide the most detailed instructions on how to conduct
comparison studies. Three categories of comparisons are distinguished into tests for non-inferiority, equivalence, and superiority.
The three categories are described in detail and graphically illustrated in CPMP's Points to Consider on Switching Between Superiority and Non-Inferiority, and only briefly summarized here. When using the CPMP guideline and one of the categories within, two important points must
be discussed and justified in the regulatory submission.
- The chosen comparison category must be explained and justified. A method comparison protocol should provide the design of
experiments to be done in a formal study and which statistical comparison test will be used, as well as a pre-specified value
for the allowable difference in results (described by the following text).
- The pre-specified maximum allowable difference (or delta) must be clearly set in the protocol, but finding and justifying
an appropriate delta is the most difficult part of any comparison study. Some guidance is provided in CPMP's Points to Consider on the Choice of Non-Inferiority Margin, London 26 February 2004, however, a clear strategy and good examples are missing. Delta should be derived as to strike a
balance between being too large (easy to pass but observed potential difference may be too large) and being too small (difficult
to pass but observed statistical difference may be overall acceptable). Delta should be derived similar to acceptance criteria
for AMV protocols that are release specifications and historical process control data should be related to delta.
Method Comparison Study Examples
The comparison tests of non-inferiority, superiority, and equivalence are described and justified in the following examples
for test method changes.
Figure 1. Results for the Non-Inferiority Test: Candidate Method vs EP/USP Sterility