FAQ/power/roc - CBU statistics Wiki

Upload page content

You can upload content for the page named below. If you change the page name, you can also upload content for another page. If the page name is empty, we derive the page name from the file name.

File to load page content from
Page name
Comment
Type the odd letters out: ONlY twO thinGs aRE infiNite

location: FAQ / power / roc

Obtaining the power for a ROC analyses

The below is from Thom Baguley (delivered to psych-postgrads jiscmail list):

Power calculations for ROC analyses can be obtained in R and are the same for apriori and post-hoc scenarios: 

install.packages('pROC')
library(pROC)
power.roc.test(ncases=41, ncontrols=72, auc=0.73, sig.level=0.05)

     One ROC curve power calculation 

         ncases = 41
      ncontrols = 72
            auc = 0.73
      sig.level = 0.05
          power = 0.9897453

The problem is where to get the true population AUC from and what it means. If you plug in an observed AUC then power is simply a monotonic function of the p value (or something close to it) and hence uninformative. (Power will be high for significant effects and low for non-significant ones regardless of the true power of the test). If you plug in the true population AUC you get the actual power, but the true value is unknown (or worse still may not be fixed but is a random variable depending on many other variables). The best practice here is either not to do it or be cautious and realise that it means if the true auc were .73 (which it almost certainly isn't) then an identical replication (which is a near-impossible ideal) would have 99% power.

Further details including formulae used are given here including a pdf link to Hanley, JA and McNeil BJ (1982) The Meaning and use of the area under a receiving operating characteristic curve. Radiology 143(1) 29. here.