A guide to obtaining confidence intervals for effect sizes
Effect sizes, specify the magnitude of a statistical comparison. However, this does not tell us how precisely it is measured.
There are details by Geoff Cumming, Keselman et al (2008) and Steiger (2004) on combining these concepts by giving confidence intervals for effect sizes. Michael Smithson has syntax in SPSS and other statistical software to do the computations. There are also some workshop notes to explain what's going on. See also Smithson (2001).
This on-line calculator will work out CIs for f-squareds which can be converted into R-squareds using the calculator here which uses the formula of R-squared = f-squared/(1+f-squared). This can be also done in R using the CI.Rsq function which uses the standard error for R-squared equal to sqrt((4*rsq*(1-rsq)2 *(n-k-1)2 )/((n2 -1)*(n+3))) for a R-squared (rsq) with k predictors and a sample size of n.
There is also SPSS syntax, with an example, for obtaining a confidence interval for Cohen's d and R syntax for an alternative more robust nonparametric bootstrap estimate (see for example Keselman et al (2008)) for confidence intervals for Cohens' d and correlations which come from non-Normal distributions.
Keselman et al (2008) have some SAS V9.1 code with examples to produce bootstrap confidence intervals for effect sizes (ie based on repeated random sampling) for a robust (winsorised) version of Cohen's d in mixed anovas (replacing the lowest and highest 20% of outcome data by their respective least extreme values. This paper is available free to CBSUers using the APA internet link.
Bird (2022) gives formulae for the comparison of pairs of standardised differences in group means.
References
Bird, KD (2022, April 11). Improved Confidence Intervals for Differences Between Standardized Effect Sizes. Psychological Methods. Advance online publication. http://dx.doi.org/10.1037/met0000494
Fidler, F and Thompson, B (2001) Computing correct confidence intervals for ANOVA Fixed- and random-effects effect sizes. Educational and Psychological Measurement 61 575-604. CIs computed using step-by-step standard output from ANOVAs.
Keselman, HJ, Algina, J, Lix, LM, Wilcox, RR, Deering, KN (2008) A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes Psychological Methods 13(2) 110-129.
Smithson, M (2001) Correct confidence intervals for various regression effect sizes and parameters: the importance of noncentral distributions in computing intervals. Educational and Psychological Measurement 61 605-632.
Steiger, JH (2004) Beyond the F Test: Effect Size Confidence Intervals and Tests of Close Fit in the Analysis of Variance and Contrast Analysis. Psychological Methods 9(2) 164-182.