Step 2: Study design and determining sample size
Table 1
|
As noted above, once the EAC is established, the adequacy of the study design to assess comparability must be evaluated. This
work is performed before the new process data are collected. Study design and sample-size calculations are critical when conducting
a statistical test. As in all statistical investigations, there is a possibility that the conclusions may be incorrect. In
the context of an equivalence test, two different errors exist. The first error takes the form of a false positive—declaring
two processes are equivalent when they are in fact not equivalent. This is called a type 1 error and represents the risk to
the consumer, because stating that the two processes are equivalent when they are not could possibly compromise efficacy or
patient safety. The second error is a false negative, which is declaring that two processes are not equivalent when they are
in fact equivalent. This is the type 2 error. The type 2 error represents the manufacturer's risk, becuase not claiming equivalence
when equivalence exists will result in unnecessary delays in the implementation of the change. Table I organizes the declarative
statement concerning equivalence relative to the true state of nature with the corresponding error type.
For an equivalence test, the type 1 error is generally set at 5%. This represents the statistical test size and is the reason
why the 90% confidence interval is computed in the testing process (5). Once the type 1 error rate is established, the desired
type 2 error is obtained by selecting an appropriate number of new lots.
One advantage of the statistical equivalence approach is that the type 1 error rate (i.e., consumer risk) is established at
the desired level and is not a function of the total sample size. Thus, the consumer is always protected at the desired level.
The type 2 error is controlled through appropriate study design. When an experimental design has an adequate number of lots,
the investigator is rewarded for proper experimental planning.
Table 2
|
For the example, the type 1 error is set at 5%. In general, if the desired type 1 error is X%, a two-sided 100-2X% confidence
interval is required in the equivalence test. The type 2 error is computed under different study designs for comparing new
lots to the four historical lots. The type 2 error is computed assuming the true difference between slopes is 0 and also when
the true difference in slopes is 25% of the EAC (0.25% per month) and 50% of the EAC (0.5% per month). Table II lists the
type 2 error rates for these different scenarios. These error rates were computed using computer simulation and the estimated
variances of the historical data.
Based on the information in Table II, four new lots provided adequate control of the type 2 error with measurements to be
taken at 0, 2, 4, and 6 months for each of the four lots.
Once the EAC and sample design have been established, it is helpful to represent the EAC graphically. Algebraically, equivalence
is demonstrated if
where bHistoric is the average slope estimate of the historical process lots, bNews is the average slope estimate of the new process lots, and ME is the margin of error for a two-sided 90% confidence interval.
The formula for the margin of error depends on the assumed statistical design. A formula is given for one particular case
in the numerical example that follows.
Equation (1) can be rewritten as
Figure 3. Average slope of the historical process with graphical representation of equivalence acceptance criteria (EAC).
Equivalence is demonstrated if new process average slope falls within red lines.
|
Figure 3 plots the average slope for the historical process (bHistoric) for the four lots shown in Figure 2 with a y-intercept equal to the average of the y-intercepts across all four lots. The
two red lines have the same y-intercept, but with slopes of (bHistoric)–EAC+ME and (bHistoric)+EAC–ME. If the average slope for the new process lots (bNews) falls within the red lines, then Equation 2 is satisfied, and equivalence of average slope is demonstrated. In this example,
ME = 0.371%, (bHistoric) = –1.13% per month, y-intercept = 89.5%, and EAC = 1% per month. For purposes of Figure 3, ME is estimated using the variance
estimates of only the historical data. The final value for ME will include variance estimates from both the historical and
new processes.
|
