402015-09-15 14:38:28PeterWatson392015-09-15 14:37:41PeterWatson382013-05-08 14:03:06PeterWatson372013-03-08 10:17:24localhostconverted to 1.6 markup362013-01-29 11:25:09PeterWatson352013-01-29 11:22:41PeterWatson342013-01-29 10:46:30PeterWatson332013-01-29 10:46:05PeterWatson322013-01-29 10:45:40PeterWatson312008-06-02 14:39:50PeterWatson302008-06-02 10:06:23PeterWatson292008-06-02 10:05:56PeterWatson282008-05-15 12:16:40IanNimmoSmith272008-05-15 12:15:52IanNimmoSmith262008-04-25 11:39:17PeterWatson252008-04-24 15:51:13PeterWatson242008-04-24 14:45:01PeterWatson232008-04-24 14:43:10PeterWatson222008-04-24 14:41:53PeterWatson212008-04-24 14:40:54PeterWatson202008-04-24 14:38:38PeterWatson192008-04-24 14:11:27PeterWatson182008-04-24 14:09:01PeterWatson172008-04-24 13:33:10PeterWatson162008-04-24 10:53:51PeterWatson152008-04-24 10:53:27PeterWatson142008-04-24 10:51:23PeterWatson132008-04-24 10:51:03PeterWatson122008-04-24 09:39:25PeterWatson112008-04-24 09:38:21PeterWatson102008-04-24 09:36:54PeterWatson92008-04-24 09:36:28PeterWatson82008-04-23 16:23:42PeterWatson72008-04-18 16:51:40IanNimmoSmith62008-04-18 16:50:28IanNimmoSmith52008-02-11 16:57:47IanNimmoSmith42008-02-11 16:48:28IanNimmoSmith32008-02-11 16:47:21IanNimmoSmith22008-02-11 16:45:45IanNimmoSmith12008-02-11 16:38:46IanNimmoSmith

The following MATLAB code may be used to perform Stouffer's method. Note the below performs Stouffer's method in R assuming p-values are entered into a vector p e.g. p <- c(0,1,0.2,0.01). A spreadsheet can also be used to compute Fisher's and Stouffer's combined p.

The basic idea is that if $$p_i (i=1 \ldots n)$$ are the one-sided $$p$$-values for $$n$$ independent statistics then $$-2 \sum\log(p_i)$$ is a $$\chi^2(2n)$$ statistic which reflects whether the combined $$p$$-values are smaller than would be expected if they were Uniform(0,1) variates. The following MATLAB code evaluates this statistic and its p-value. Let's try it out: > pvals=[0.1 0.01 0.01 0.7 0.3 0.1]; >> pfast(pvals) ]]>I.e. the combined p-value is 0.0021 for this array of 6 $$p$$-values. Further investigations suggest that Fisher's method has inappropriate behaviour. [examples to be included] This method may also be performed using R code. van Assen, van Aert and Wicherts (2015) give a formula based upon Fisher's method for summing p-values from studies a meta-analysis and comparing this sum to a Gamma distribution to assess for publication bias. Manolov R and Solanas A (2012) suggest performing a binomial test to see if more statistically significant results (p < alpha) occur than would be expected assuming the null probability of a significant result is alpha (e.g. 0.05). The test can be evaluated routinely performed in most packages. For example, suppose as in Manolov and Solanas (2012, p.505) 3 out of 10 studies, or cases, have a p-value less than 0.05 for the same test statistic. We can input a single column (count) with 3 '1's and 7 '0's and compare this to a binomial test with P(a significant result)=0.05 denoted by 'testvalue' in the below syntax (rather than the default of 0.5). This can be done using the SPSS syntax below. This gives a p=0.012 agreeing with the value on page 505 of Manolov and Solanas. That is, the probability of obtaining equal or more extreme numbers of p-values than that observed equals P(three or fewer p-values < 0.05) + P([10-3=] seven or more p-values < 0.05) = 0.012.

Bailey TL and Gribskov M (1998). Combining evidence using p-values: application to sequence homology searches. Bioinformatics, 14(1) 48-54. Fisher RA (1925). Statistical methods for research workers (13th edition). London: Oliver and Boyd. Manolov R and Solanas A (2012). Assigning and combining probabilities in single-case studies. Psychological Methods 17(4) 495-509. Describes various methods for combining p-values including Stouffer and Fisher and the binomial test. Stouffer, Samuel A., Edward A. Suchman, Leland C. DeVinney, Shirley A. Star, and Robin M. Williams, Jr. (1949). Studies in Social Psychology in World War II: The American Soldier. Vol. 1, Adjustment During Army Life. Princeton: Princeton University Press. van Assen MALM, van Aert RCM and Wicherts JM (2015) Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods 20(3) 293-309.