Exact test vs chi-square?

Howell (2002) on p.158 explains a difficulty using the Pearson chi-square test for testing the independence of frequencies in two-way tables. Namely that when the expected cell frequencies are less than five the Pearson statistics does not follow a chi-square distribution.

Instead it is recommended that Fisher's exact test is used when any expected frequency is less than five.

[wiki:FAQ/ChiSquaredSpss CROSSTABS] in SPSS, consequently, outputs how many expected cell frequencies are less than five whenever a chi-square test is performed. Fisher's exact test may be requested by pressing on the exact button in the crosstabs dialogue box and requesting 'exact'. Alternatively the [wiki:SoftwareNotes locally available software] program fishrc computes this test on a UNIX machine.

The p-value can then be quoted. This p-value is always two-sided except in the case of a 2 by 2 table where it can also be one-sided.

For EXCEL users the BINOMDIST function can be used for exact probabilities for the special case of a single proportion (equivalently a 2 by 1 table) using its cumulative distribution function.

For example BINOMDIST(11,23,0.5,TRUE) = the probability of observing at most 11 'correct' out of a possible 23 assuming the chance of observing a correct is equal to a half = 0.5 (since 11 is the midpoint of the distribution and the binomial distribution is symmetric).

Reference

Howell DC (2002) Statistical methods for psychology fifth edition: Duxbury Press:Pacific Grove, CA.