FAQ/effcis - CBU statistics Wiki

Upload page content

You can upload content for the page named below. If you change the page name, you can also upload content for another page. If the page name is empty, we derive the page name from the file name.

File to load page content from
Page name
Comment
Finzd thee wrang lelters ino eacuh wosrd

location: FAQ / effcis

A guide to obtaining confidence intervals for effect sizes

Effect sizes, specify the magnitude of a statistical comparison. However, this does not tell us how precisely it is measured.

There are details by Geoff Cumming, Keselman et al (2008) and Steiger (2004) on combining these concepts by giving confidence intervals for effect sizes. Michael Smithson has syntax in SPSS and other statistical software to do the computations. There are also some workshop notes to explain what's going on. See also Smithson (2001).

This on-line calculator will work out CIs for f-squareds which can be converted into R-squareds using the calculator here which uses the formula of R-squared = f-squared/(1+f-squared). This can be also done in R using the CI.Rsq function which uses the standard error for R-squared equal to sqrt((4*rsq*(1-rsq)2 *(n-k-1)2 )/((n2 -1)*(n+3))) for a R-squared (rsq) with k predictors and a sample size of n.

There is also SPSS syntax, with an example, for obtaining a confidence interval for Cohen's d and R syntax for an alternative more robust nonparametric bootstrap estimate (see for example Keselman et al (2008)) for confidence intervals for Cohens' d and correlations which come from non-Normal distributions.

Keselman et al (2008) have some SAS V9.1 code with examples to produce bootstrap confidence intervals for effect sizes (ie based on repeated random sampling) for a robust (winsorised) version of Cohen's d in mixed anovas (replacing the lowest and highest 20% of outcome data by their respective least extreme values. This paper is available free to CBSUers using the APA internet link.

* A suggestion for directly computing the s.e. of Cohen's d for use in constructing confidence intervals

Bird (2022) gives formulae for the comparison of pairs of standardised differences in group means.

References

Bird, KD (2022, April 11). Improved Confidence Intervals for Differences Between Standardized Effect Sizes. Psychological Methods. Advance online publication. http://dx.doi.org/10.1037/met0000494

Fidler, F and Thompson, B (2001) Computing correct confidence intervals for ANOVA Fixed- and random-effects effect sizes. Educational and Psychological Measurement 61 575-604. CIs computed using step-by-step standard output from ANOVAs.

Keselman, HJ, Algina, J, Lix, LM, Wilcox, RR, Deering, KN (2008) A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes Psychological Methods 13(2) 110-129.

Smithson, M (2001) Correct confidence intervals for various regression effect sizes and parameters: the importance of noncentral distributions in computing intervals. Educational and Psychological Measurement 61 605-632.

Steiger, JH (2004) Beyond the F Test: Effect Size Confidence Intervals and Tests of Close Fit in the Analysis of Variance and Contrast Analysis. Psychological Methods 9(2) 164-182.