Statistical tests of equivalence

Wellek (2003) illustrates the application of a series of familiar statistical tests corresponding to null and alternative statistical hypotheses of general form

H0: $$\theta \leq $$-t or $$\theta \geq$$ t and HA : -t $$ \leq \theta \leq$$ t

where $$\theta$$ is a function of parameters of interest (e.g. a difference between two group means) and t is the effect size of minimal interest (e.g. minimum difference in a pair of group means which is of clinical interest). Equivalence testing can also, more simply, be thought of as seeing if an apriori specified clinically meaningful difference is contained in the confidence interval for the effect size obtained from the observed data. An illustration of using a confidence interval for the effect size, Cohen's d, to perform an equivalence test for two groups is is given here.

Equivalence tests are also known as reverse tests because they switch around the 'usual' hypotheses of form

H0: $$\theta$$ = t and HA: $$\theta \ne$$ t

and so the emphasis is on verifying rather than rejecting hypotheses such as equality of group means or zero correlations. Failing to reject a null hypothesis is not the same as showing it to be valid. If H0 takes just one-sided e.g. $$\theta \leq $$-t these tests are known as either inferiority or superiority tests as one is wishing to see if the confidence interval contains a signed difference showing a particular treatment is better (superior) or worse (inferior) to the other treatment. A web calculator using results from Julious (2004), Pocock (1983) and Blackwelder (1982) for equivalence, inferiority and superiority sample size calculation may be used here. These results together with those from Thabane can also be obtained using this spreadsheet.

SAS and FORTRAN programs with help guides are available for free download which run equivalence analyses for other statistical tests using methodology described in Wellek (2003). It is easier to run the SAS programs. After downloading change the file name from *.sas to *.sss before clicking on the icon. CBSUERS: If SAS is not on your PC it can be added on by one of our CBSU IT people. There is also a suite of equivalence programs for use with R.

• Suggested effect sizes, t

• R code for equivalence test for (un)paired t-tests

• R code for equivalence test for one sample z-test

• R code for equivalence test for two unrelated proportions

• R code for equivalence test using McNemar's test

• R code for equivalence test for one-way ANOVA

This pdf file comprising of class notes by Lehana Thabane of McMaster University gives simple formulae for working out samples sizes for equivalence of form difference in means, regardless of direction, being at least delta and one-sided versions - differences in mean 1 - mean 2 <= delta (Superiority) and mean 1 - mean 2 >= delta (Inferiority). The Thabane formulae correspond to the non-inferiority examples in Julious's website mentioned above.

References

Blackwelder WC (1982) Proving the Null Hypothesis" in Clinical Trials. Control. Clin. Trials 3 , 345-353.

Donner A (1984) Approaches to sample size estimation in the design of clinical trials - a review. Statistics in Medicine 3 199-214. Looks at formulae for equivalence and sample size formulae for power analyses.

Julious SA (2004) Sample sizes for clinical trials with Normal data. Statistics in Medicine 23, 1921-1986.

Lew MJ (2006) Principles: When there should be no difference - how to fail to reject the null hypothesis. Trends in Pharmacological Sciences 27(5) 274-278. Available to CBSU users on ScienceDirect.

Pocock SJ (1983) Clinical Trials: A Practical Approach. Wiley.

Wellek S (2003) Testing of statistical hypotheses of equivalence. Chapman and Hall/CRC Press.