Combining p-values by Fisher's method and Stouffer's method
The basic idea is that if $$p_i (i=1 \ldots n)$$ are the one-sided $$p$$-values for $$n$$ independent statistics then $$-2 \sum\log(p_i)$$ is a $$\chi^2(2n)$$ statistic which reflects whether the combined $$p$$-values are smaller than would be expected if they were Uniform(0,1) variates.
The following MATLAB code evaluates this statistic and its p-value.
function p = pfast(p)
% Fisher's (1925) method for combination of independent p-values
% Code adapted from Bailey and Gribskov (1998)
product=prod(p);
n=length(p);
if n<=0
error('pfast was passed an empty array of p-values')
elseif n==1
p = product;
return
elseif product == 0
p = 0;
return
else
x = -log(product);
t=product;
p=product;
for i = 1:n-1
t = t * x / i;
p = p + t;
end
end Let's try it out:
>> pvals=[0.1 0.01 0.01 0.7 0.3 0.1];
>> pfast(pvals)
ans =
0.0021I.e. the combined p-value is 0.0021 for this array of 6 $$p$$-values.
Further investigations suggest that Fisher's method has inappropriate behaviour. [examples to be included]
Combining p-values by Fisher's method
function pcomb = stouffer(p)
% Stouffer et al's (1949) unweighted method for combination of
% independent p-values via z's
if length(p)==0
error('pfast was passed an empty array of p-values')
pcomb=1;
else
pcomb = (1-erf(sum(sqrt(2) * erfinv(1-2*p))/sqrt(2*length(p))))/2;
end
References
Bailey TL, Gribskov M (1998). Combining evidence using p-values: application to sequence homology searches. Bioinformatics, 14 (1) 48-54.
Fisher RA (1925). Statistical methods for research workers (13th edition). London: Oliver and Boyd.
Stouffer, Samuel A., Edward A. Suchman, Leland C. DeVinney, Shirley A. Star, and Robin M. Williams, Jr. (1949) Studies in Social Psychology in World War II: The American Soldier. Vol. 1, Adjustment During Army Life. Princeton: Princeton University Press.