Two effect sizes: Kendall's W and average Spearman's rho

Two measures akin to correlation have been proposed as effect sizes Howell, 1997; Hays, 1981) when looking at agreement between raters using ranking procedures such as Friedman's test.

Kendall’s W is may be outputted in SPSS when computing Friedman’s test in analyse>nonparametric etsts>k related samples.

Kendall's W = X/(n(k-1)) where X = Friedman's chi-square, n = the number of cases (subjects, raters)and k = the number of conditions (objects) (i.e., the data are n sets of ranks of k things). 0 <= W <= 1.

The average Spearman r among the n cases is rbar = (n*W-1)/(n-1). -1/(n-1) <= rbar <= 1.

If the test statistic W is 1, then all the survey respondents have been unanimous, and each respondent has assigned the same order to the list of concerns. If W is 0, then there is no overall trend of agreement among the respondents, and their responses may be regarded as essentially random. Intermediate values of W indicate a greater or lesser degree of unanimity among the various responses.

Further details are in Howell DC (1997) in a chapter entitled ‘Alternative Correlational Techniques’ at least in the fourth edition! Howell follows Hayes (1981) in suggesting converting Kendall’s W to the average Spearman r as described above.

References

Hays WL (1981) Statistics 3rd Edition. Holt, Rinehart and Winston:New York

Howell DC (1997) Statistical methods for psychology. Fourth Edition. Duxbury Press:Belmont, CA.