I am in the process of conducting a meta-analysis. I am having difficulty replicating effect sizes that are in previous meta-analyses on the same topic. That is, for a given study I am not getting the same effect size estimate.

In these past meta-analyses, effect sizes and p-values are provided. Is it possible to use these estimates to obtain the variance of the effect size. For example, can I use the p-value and effect size to obtain the 95 % confidence interval, through which I can obtain the standard error.

As an aside, I have reached out to the authors of the previous meta-analyses and they have not responded.

  • $\begingroup$ What kind of effect size measure are you referring to? Raw or standardized mean differences? Log odds/risk ratios or risk differences? Something else? Also, just to clarify: You have a bunch of these estimates based on individual studies, and corresponding p-values, and you want to compute the variances (standard errors) thereof? $\endgroup$ – Wolfgang Apr 10 '16 at 10:41
  • $\begingroup$ Hello. I have standardized mean differences, both Cohen's d and Hedges G. Indeed, I have many effect size estimates with corresponding p-values and would like to calculate the variance for each effect. $\endgroup$ – D_Williams Apr 10 '16 at 15:32
  • $\begingroup$ In addition to effect size estimates and p-values, I also have the sample sizes. $\endgroup$ – D_Williams Apr 10 '16 at 15:58
  • $\begingroup$ Sample sizes of each group or just total (both groups combined)? $\endgroup$ – Wolfgang Apr 10 '16 at 16:46
  • $\begingroup$ In one of the meta-analyses, the total is provided. In a different meta-analyses there are sample sizes for each group. From the original studies, however, I can obtain the sample sizes for each group. $\endgroup$ – D_Williams Apr 10 '16 at 18:10

If you know the standardized mean difference ($d$) for a study and the sample sizes of the two groups ($n_1$ and $n_2$), then you can compute (or to be precise: estimate) the sampling variance with: $$v = \frac{1}{n_1} + \frac{1}{n_2} + \frac{d^2}{2(n_1 + n_2)}.$$ If you only know the total sample size of a study ($N$), you could assume $n_1 = n_2 = N/2$. The square-root of the sampling variance then gives you the standard error of $d$.

A bias-correction is often applied to $d$ values. Roughly, the bias-corrected standardized mean difference (often called Hedges' g) is given by: $$g = \left(1 - \frac{3}{4m-1} \right)d,$$ where $m = n_1 + n_2 - 2$ (this may explain the discrepancy you found when you computed the standardized mean difference yourself versus what was reported in the paper). Then you can plug $g$ into the equation for the sampling variance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.