This is my question: I have to perform a mini-meta-analysis on two studies(independent, balanced sample) with a Repeated measures design and with two within factors. I have the F statistic and the p value of two significant interactions in study 1 and study 2. I want to present a unique p value, so I have to combine the p values. Is it correct to transform the p value in z score and after do I have to average the z scores and find the correspondent p value? Can you suggest a paper with a similar procedure?


There is a paper by Zaykin which discusses this and also extends Stouffer's method to incorporate weights. Optimally weighted Z-test is a powerful method for combining probabilities in meta-analysis. Journal of Evolutionary Biology 24:1836-1841, 2011

Basically you form

${\frac{\sum (w z(p))}{\sqrt {\sum w ^ 2}}}$

where $w$ are the weights, $p$ the p-values, $z()$ is the normal deviate. In the absence of effect sizes (in which case a method for combining effect sizes woud be more appropriate anyway) best results are believed to be obtained with weights proportional to the square root of the sample sizes (according to Zaykin).

  • $\begingroup$ Just a clarification, what do you mean for normal deviate? is it the the z score obtained from the p values? Sorry, I am completely new to this procedure. $\endgroup$ – chiara Feb 15 '18 at 15:28
  • $\begingroup$ Yes, that is what I meant. $\endgroup$ – mdewey Feb 15 '18 at 17:08
  • $\begingroup$ Thanks! To re-transform the combined z score in p value, should I use as df (N-k), where N is the ∑n1+n2 and k the number of the studies? $\endgroup$ – chiara Feb 16 '18 at 10:21
  • $\begingroup$ N is sample size $\endgroup$ – chiara Feb 16 '18 at 10:29
  • $\begingroup$ For the normal there is no concept of degrees of freedom so not sure what you are asking. $\endgroup$ – mdewey Feb 16 '18 at 11:47

Michael Strube has a paper about this titled as "Combining and Comparing significance Levels From Nonindependent Hypothesis Tests" (1985). What you describe is pretty similar to what he says; the only thing is that you don't take the average of the z scores but you should divide the sum of the two z scores to square root of 2.

  • $\begingroup$ Many thanks to both of you, very important answers to me. $\endgroup$ – chiara Feb 15 '18 at 14:16
  • $\begingroup$ Hello, I am trying to apply the abovementioned formula: I have a p=.001 and a sample of 25 pps, to which it should correspond a Z of 2.43, I have also a second sample of 33 pps with a p of .13 to which it should correspond a Z of 1.13. But I get a weird result. Can you give some tips? $\endgroup$ – chiara Oct 24 '19 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.