Current Regulatory Guidance Documents

Unfortunately, there is currently no clear guidance available on how to demonstrate the equivalence of the candidate method. Several guidance documents contain general instructions that are helpful but do not provide sufficient detail on which characteristic to compare and how to compare them. Also, the scope and intention of some guidelines were not designed for the purpose of comparing a candidate method's performance to the current one. However, the following documents provide some guidance for method comparison and/or use of statistical tests:

ISPE, Good Practice Guide on Technology Transfer, 2003.

CPMP, Points to Consider on Switching Between Superiority and Non-Inferiority, London, 27 July 2000.

CPMP, Points to Consider on the Choice of Non-Inferiority Margin, London, 26 February 2004)

PDA's Technical Report No. 33., Evaluation, Validation and Implementation of New Microbiological Testing Methods.

ASTM International's D4855-97, Standard Practice for Comparing Methods.

All of the guidance documents listed can be used to establish a firm's standard procedure for method comparability studies. These and possibly other general guidelines constitute good base references if followed as written, and provide a good regulatory submission tool.

Demonstrating Non-Inferiority, Equivalence, and Superiority

Of all guidance documents described, the ICH E9 and CPMP guidelines provide the most detailed instructions on how to conduct comparison studies. Three categories of comparisons are distinguished into tests for non-inferiority, equivalence, and superiority. The three categories are described in detail and graphically illustrated in CPMP's Points to Consider on Switching Between Superiority and Non-Inferiority, and only briefly summarized here. When using the CPMP guideline and one of the categories within, two important points must be discussed and justified in the regulatory submission.

• The chosen comparison category must be explained and justified. A method comparison protocol should provide the design of experiments to be done in a formal study and which statistical comparison test will be used, as well as a pre-specified value for the allowable difference in results (described by the following text).
• The pre-specified maximum allowable difference (or delta) must be clearly set in the protocol, but finding and justifying an appropriate delta is the most difficult part of any comparison study. Some guidance is provided in CPMP's Points to Consider on the Choice of Non-Inferiority Margin, London 26 February 2004, however, a clear strategy and good examples are missing. Delta should be derived as to strike a balance between being too large (easy to pass but observed potential difference may be too large) and being too small (difficult to pass but observed statistical difference may be overall acceptable). Delta should be derived similar to acceptance criteria for AMV protocols that are release specifications and historical process control data should be related to delta.

Method Comparison Study Examples

Figure 1. Results for the Non-Inferiority Test: Candidate Method vs EP/USP Sterility