I am doing a meta-analysis of means. I have gathered means, SD and N from a set of studies. I do not have access to the raw data of the means. After doing an Anderson-Darling test, I found that the means are not normally distributed. Thus, I log-transformed the means. But what do I do with the SD? I am using the escalc and rma functions from the package metafor in R. If I use log(mean) and log(SD) in escalc, this provides negative values of vi, which does not work in rma. Is there another way to do this? Or can I leave the SD untransformed, and just use log(mean)? The plot looks a bit strange though, as the variation of each study becomes very large.


Mean basal metabolic rate of bats

  • 1
    $\begingroup$ Unfortunately the log of a mean is not much use even if logarithmic scales make sense, and the log of an SD even less use. But negative logarithms just mean original values < 1 and it's hard to see that as invalid in general. $\endgroup$ – Nick Cox Feb 25 '20 at 16:26
  • 2
    $\begingroup$ I would suggest using the raw values you have and then looking at the diagnostics from the model using fit.rma() from the package you are using. This would give you a handle on how much the residuals deviate from normality and whether that is likely to matter. $\endgroup$ – mdewey Feb 25 '20 at 17:05
  • 2
    $\begingroup$ A more informative plot can be obtained with qqnorm() $\endgroup$ – mdewey Feb 25 '20 at 17:16
  • 2
    $\begingroup$ You should add the qqnorm plot, but that histogram doesn't look terribly non-normal, it has some skew, but do not have heavy tails. $\endgroup$ – kjetil b halvorsen Feb 25 '20 at 17:20
  • 1
    $\begingroup$ I agree with others, I think, and see no reason not to use those results as they come. Sure, the A-D test says that they are not exactly normal, but that wouldn't worry me. $\endgroup$ – Nick Cox Feb 25 '20 at 17:45

The standard meta-analytic models do NOT assume that the marginal distribution of the observed outcomes is normal.

Take, the random-effects model, which is given by $$y_i = \mu + u_i + e_i,$$ where $y_i$ is the observed outcome in the $i$th study (in your case, a mean), $u_i \sim N(0, \tau^2)$ is a random effect for heterogeneity, and $e_i \sim N(0, v_i)$ is the sampling error in the $i$th study with (approximately) known sampling variance $v_i$ (for means, the sampling variance is $v_i = \mbox{SD}^2_i / n_i$, where $\mbox{SD}_i$ is the observed standard deviation and $n_i$ the sample size).

The marginal distribution of the observed $y_i$ values (which is what you are examining) is a mixture distribution of normals with different variances and hence is not normal. Therefore, examining the distribution of the observed outcomes is not a useful diagnostic tool.

Instead, one should consider/examine the two normality assumptions underlying this model separately.

The first pertains to the sampling distributions. Based on the central limit theorem, we know that the sampling distribution of a mean will tend towards normality with increasing sample size. Hence, assuming $n_i$ is not too small in any of the studies, I wouldn't worry too much about this assumption. You cannot really say much about this assumption anyway without access to the raw data (e.g., if the raw data come from a normal distribution, then the sampling distribution of the mean will be normal no matter what the sample size is, but convergence to a normal sampling distribution could be slow (or even not happen at all) for other distributions underlying the raw data).

The second normality assumptions pertains to the $u_i$ values. One cannot observe these values directly, but we can obtain BLUPs (best linear unbiased predictions) of these values. Since you mentioned using the metafor package, you can use the ranef() function on the fitted model object to obtain those BLUPs. But even examining the distribution of the BLUPs is problematic, because they have heteroscedastic standard errors as well. One could divide the BLUPs by their standard errors and examine the distribution of those (approximate) z-values, but in my experience that only works so-so.

Finally, one can also examine the distribution of the standardized residuals (rstandard()) or studentized residuals (rstudent()). This doesn't differentiate between the two issues above, but can work as a rough check of the assumptions underlying this model in general.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.