Meta-Analysis: Effect size calculation - p-value Is it possible to calculate an effect size if only a p-value and sample sizes of two independet groups are given?
Kind regards
Joanne
 A: Assuming the p-value you have is based on an independent samples t-test, then you can do the following:


*

*If the p-value is for a two-sided test, divide the p-value by 2, so it becomes a one-sided p-value. For example, if the two-sided p-value is .046, then the one-sided $p = .023$.

*Convert the one-sided p-value to the corresponding t-statistic. The degrees of freedom for the t-distribution are $n_1 + n_2 - 2$, where $n_1$ and $n_2$ are the group sizes. For example, if the (one-sided) p-value is .023 and $n_1 = 23$ and $n_2 = 27$, then $t = 2.048508$. One can use tables for this lookup or use software like R (e.g., qt(.023, df = 23+27-2, lower.tail = FALSE) in R).

*Convert the t-statistic to Cohen's d with: $$d = t \times \sqrt{1/n_1 + 1/n_2}.$$ So, continuing the example, this would be: $$d = 2.048508 \times \sqrt{1/23 + 1/27} = 0.5812686.$$

*Convert Cohen's d to Hedges' g by applying the bias-correction: $$g = \left(1 - \frac{3}{4(n_1 + n_2 - 2) - 1}\right) \times d.$$ So, in the example: $$g = \left(1 - \frac{3}{4(23 + 27 - 2) - 1}\right) = 0.5721388,$$ which we then can round to 0.57.


Notes:


*

*Again, this conversion of a p-value to Cohen's d / Hedges' g only makes sense when the p-value comes from an independent samples t-test.

*Moreover, the independent samples t-test must have been a t-test assuming equal variances in the two groups (the 'homoscedasticity assumption'). If the t-test was a Welsh's t-test, then this conversion is not appropriate.

*If the p-value was two-sided to begin with, you do not know whether the sign of the d or g value should be positive or negative. So, the two-sided p-value of .046 above corresponds to $g = 0.57$ or $g = -0.57$. You will need further information (from the article/source from which the p-value has come) to determine the correct sign.

*The conversion is in principle exact, but since reported p-values are often rounded, this will introduce some error into the computations. That's usually not a major concern though.

*However, if the p-value is truncated (e.g., all you know is that $p \le .05$ or some other cutoff), then this is a problem. Some people might assume $p = .05$ then (which will be overly conservative), others might assume that the p-value is then halfway between 0 and the cutoff, so $p = .025$ (which will be conservative or liberal depending on what the exact p-value actually was), and some people might then just decide not to convert the p-value to d or g due to insufficient information.

