More on the Hayter-Fisher multiple comparison test

An explanation of the procedure:

Whereas the Tukey HSD and Tukey-Kramer are well known, the Hayter-Fisher test is relatively new (Hayter, 1986). Like the Tukey or Tukey-Kramer test, it controls alpha familywise at .05 for all the pairwise comparisons in an experiment. Unlike the Tukey/Tukey-Kramer tests, the Fisher-Hayter method needs the overall F test to be significant in order to maintain control of alpha familywise. Because most researchers use the overall F test before doing multiple comparison tests, it is easy to use the Hayter-Fisher. The advantage of Hayter-Fisher over Tukey is that it gives a slight gain in power. The Hayter-Fisher method uses a smaller critical value of the studentised range statistic than both the Tukey and Tukey-Kramer tests so has a smaller p-value.

If the sample sizes differ substantially Toothaker and Miller (1996) suggest using the Games-Howell procedure.

References

Hayter, A.J. (1986). The maximum familywise error rate of Fisher's least significant difference test. Journal of the American Statistical Association, 81, 1000-1004.

Toothaker, L. E., & Miller, L. (1996). Introductory Statistics for the Behavioral Sciences, 2nd Ed. Pacific Grove, CA: Brooks/Cole. Further details of the Hayter-Fisher computation can be found in this reference.