Cohen's d pooling across subjects
Suppose we have responses measured on different subjects at each of two time points and wish to obtain a pooled Cohen's d for the difference between these time points.
One approach is to firstly compute Cohen's d for each case and then average these.
Cohen’s d for each case = difference in group means for that case divided by the pooled group within subject SD for that case
where the pooled group within subject SD for that case is the weighted average of the variances weighted by the sample size in each group. If the groups are equal the pooled SD equals
the square root of [(SD2 for that case in group 1 + SD2 for that case in group 2)/2]
This seems a reasonable approach because we are using an SD based upon the within subject variances as our measure of SD for means which are also within subject to compute Cohen’s d.
Arntz et al (2013) use a more complex alternative pooled estimate using a multilevel model to separate out the between and within subject error variances. Suppose we wish to estimate the difference between an intervention and baseline mean (in a group factor, intervention) with upto six repeated assessments in a variable time taken either at baseline or intervention. An example SPSS data set is is here. The below fits the model of Arntz et al (2013) in SPSS accounting for the correlation structure of the time points within each subject. We obtain the fixed effect regression estimate for the intervention from this analysis.
MIXED y BY intervention with time /FIXED =time intervention /METHOD=REML /RANDOM = intercept | SUBJECT(id) COVTYPE(id) /REPEATED = time | SUBJECT(id) COVTYPE(arma11) /PRINT = COVB G HISTORY SOLUTION TESTCOV.
Arntz et al then fit a null model which contains no predictors and obtain the within subject variance and the between subject variance.
MIXED y /METHOD=REML /RANDOM = intercept | SUBJECT(id) COVTYPE(id) /PRINT = COVB G HISTORY SOLUTION TESTCOV.
Adding these two variances together and square rooting gives an estimate of the pooled standard deviation pooled over both time points and subjects. Dividing the intervention effect from the first analysis by this pooled SD then gives a pooled estimate of Cohen's d.
Reference
Arntz, A, Sofi, S and van Breukelen, G (2013). Imagery Rescripting as treatment for complicated PTSD in refugees: A multiple baseline case series study. Behaviour Research and Therapy 51 274-283.