Comparing all pairwise comparisons in an anova assuning equality of group variances

Ramsey PH and Ramsey PP (2008) recommend using the Tukey-Kramer procedure to compare all possible group means assuming homogeneity of group variances. It is found to give the best any-pair power if the overall F test is not significant. This procedure is computed for upto 10 means using this [attachment:mcneq+hf.xls spreadsheet.] It may also be computed using the mc or mcneq procedures found by typing mc and mcneq at a UNIX prompt. Ramsey and Ramsey (2008) further recommend using the more conservative Hayter-Fisher modification of the Tukey-Kramer procedure for maximizing any-pair power in the presence of a significant overall F value. This is also computed in the spreadsheet.

The Games-Howell approach is recommended for all pairwise comparisons when there are heterogeneous group variances. The Games-Howell approach may be computed in SPSS using the GLM:univariate procedure or by using this [attachment:gamesh.xls spreadsheet.]

A flow chart detailing post-hoc tests for between subject designs is [attachment:flow.pdf here.]

Reference

Ramsey PH and Ramsey PP (2008) Power of pairwise comparisons in the equal variance and unequal sample size case. British Journal of Mathematical and Statistical Psychology 61(1) 115-131.