Exact test vs chi-square?

Howell (2002) on p.158 explains a difficulty using the Pearson chi-square test for testing the independence of frequencies in two-way tables. Namely that when the expected cell frequencies are less than five the Pearson statistics does not follow a chi-square distribution.

Instead it is recommended by Howell that Fisher's exact test is used when any expected frequency is less than five although Pett (1997) suggests problems using alternatives to chi-square tests if fewer than a fifth of expected values are less than five. Campbell ( )

• here mentions that the exact test is too conservative for 2x2 tables and suggests using the linear-by-linear chi-square test produced in SPSS CROSSTABS which is also available using his on-line calculator at this website.

CROSSTABS in SPSS, consequently, outputs how many expected cell frequencies are less than five whenever a chi-square test is performed. Fisher's exact test may be requested by pressing on the exact button in the crosstabs dialogue box and requesting 'exact'. Alternatively the locally available software program fishrc computes this test on a UNIX machine.

The p-value can then be quoted. This p-value is always two-sided except in the case of a 2 by 2 table where it can also be one-sided.

For EXCEL users the BINOMDIST function can be used for exact probabilities for the special case of a single proportion (equivalently a 2 by 1 table) using its cumulative distribution function.

For example BINOMDIST(11,23,0.5,TRUE) = the probability of observing at most 11 'correct' out of a possible 23 assuming the chance of observing a correct is equal to a half = 0.5 (since 11 is the midpoint of the distribution and the binomial distribution is symmetric).

Reference

Howell DC (2002) Statistical methods for psychology fifth edition: Duxbury Press:Pacific Grove, CA.

Pett MA (1997) Nonparametric statistics for Health Care Research. Sage publications:London.