FAQ/nonpz - CBU statistics Wiki
Self: FAQ/nonpz

A two group nonparametric effect size

Field (2005) suggests an effect size associated with performing nonparametric group tests. This is analogous to Cohen's d for parametric group testing. In particular on pages 531-2 of his book Field suggests using the formula of Rosenthal (1991, p.19) to compute a correlation:

correlation = z /sqrt(N) 

where z is the z statistic that SPSS produces when you do the Mann-Whitney or Wilcoxon tests under analyze>nonparametrics and N is the size of the study. This correlation can then be converted to a Cohen's d as required using Jamie DeCoster's (2012) EXCEL spreadsheet here) which uses results in Rosenthal (1994).

The z statistic is outputted by default using SPSS and can be obtained in R using the qnorm() procedure as in the below for response, y, with two group membership vector, group.

wtest <- wilcox.test(y~group,data=d, alternative="greater")
qnorm(wtest$p.value)

This is a correlation so can be compared to the rules of thumb suggested by Cohen with suggested thresholds of 0.1, 0.3 and 0.5 for small, medium and large magnitudes respectively. In fact the above statistic is equal to the Phi Coefficient (Howell, 1997,2001,2006) which is used to generate a correlation from 2x2 frequency tables and is of form

Sqrt(chi2 /N) = Sqrt(z2 /N) = z /sqrt(N)

This effect size can be worked out for the Mann-Whitney test (or Wilcoxon test with the two groups representing increases and decreases and putting as the response the unsigned change scores for each subject) using this spreadsheet. The input is the total number of subjects, group (coded as either 1 or 2) and the response. For paired responses input the paired difference for each subject as the response.

There are similar (and easily calculated) alternatives recommended for two-group nonparametric effect sizes - see e.g. Newcombe (2006).

You can also use freeware to compute the z-value used above using nonparametric tests and other analyses from here clicking on 'Statistical Analyses'.

We can alternatively compute sample sizes required using a t-test then inflate this sample size per group based on the asymptotic relative efficiency of 3/π (approximately 95%), where π = Pi = 3.14, of the Mann-Whitney compared to the t-test, as reported in Lehmann (1975): so for example if n=23 per group is outputted as having appropriate power for an unpaired t-test the Mann-Whitney test would require 23/(3/π)≈24 per group.

Three or more groups

Effect sizes comparing three or more groups are more difficult to interpret and risk omitting an important pairwise difference so effect sizes such as the above based upon pairwise group comparisons are recommended (Field (2005)). Field, however, suggests a pooled group estimate yielding a z-value for insertion in the above effect size as suggested here.

References

DeCoster J (2012) Spreadsheet for converting effect size measures. Available from: http://www.stat-help.com/spreadsheets/Converting%20effect%20sizes%202012-06-19.xls (accessed 04.09.2014)

Field A (2005) Discovering statistics using SPSS. Second Edition. Sage:London.

Howell DC (1997), (2001), (2006) Statistical methods for psychology. Fourth, Fifth and Sixth Editions. Wadsworth:Belmont,CA.

Lehmann, EL (1975) Nonparametrics: Statistical Methods Based on Ranks. Holden-Day, San Francisco.

Newcombe RG (2006) Confidence-intervals for an effect size measure based on the Mann-Whitney statistic. Part 1:General issues and tail-area-based methods. Statistics in Medicine 25 543-557.

Rosenthal R (1991) Meta-analytic procedures for social research (revised). Sage:Newbury Park,CA.

Rosenthal R (1994) Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The Handbook of Research Synthesis. New York, NY: Sage. pp. 239.

None: FAQ/nonpz (last edited 2016-02-11 16:06:27 by PeterWatson)