3
$\begingroup$

I am working on a meta-analysis of standardized mean differences for which researchers report a variety of unadjusted data and data that are adjusted for one or two covariates. In this research literature, the extent to which including these covariates affects the reported effect size is a matter of much debate, so it would be substantively interesting to know the extent to which adjustment for covariates affects the estimated meta-analytic effect size. Consequently, I would like to fit a model that estimates $d' - d$, where $d'$ is the covariate-adjusted effect size and $d$ is the unadjusted effect size.

Let's make the following assumptions for this question --

  1. Each researcher measures $y$ for each of two groups. The variable indicating group membership is $x$.
  2. Some researchers also measure a covariate $cov$. This covariate is common across all studies where $cov$ is measured.
  3. I have access to any statistical information required to estimate the meta-analytic model that gives me $d' - d$ (e.g., for each study, let's assume I have the means and standard deviations of $y$ and $cov$ in each group denoted by $x$, and let's additionally assume that I have access to the full variance-covariance matrix for $y$, $x$, and $cov$).

Given the assumptions above, how might I fit a model that would give me $d$, $d'$, and $d' - d$? I feel like this should be possible using multivariate meta-analysis, but I can't quite figure out what the model would look like.

Here are some sample data illustrating the question.

set.seed(214324)
# Studies without covariates
d1 <- data.frame(x = rep(c(-.5, .5), 20), y = .5 * rep(c(-.5, .5), 20) + rnorm(40, sd = 1)) 
d2 <- data.frame(x = rep(c(-.5, .5), 20), y = .5 * rep(c(-.5, .5), 20) + rnorm(40, sd = 1)) 
d3 <- data.frame(x = rep(c(-.5, .5), 20), y = .5 * rep(c(-.5, .5), 20) + rnorm(40, sd = 1)) 
d4 <- data.frame(x = rep(c(-.5, .5), 20), y = .5 * rep(c(-.5, .5), 20) + rnorm(40, sd = 1)) 
d5 <- data.frame(x = rep(c(-.5, .5), 20), y = .5 * rep(c(-.5, .5), 20) + rnorm(40, sd = 1)) 
# Studies with covariates
d6 <- data.frame(x = rep(c(-.5, .5), 20), cov = rnorm(40, sd = 1))
d6$y <- .5 * d6$x + .6 * d6$cov + rnorm(40, sd = 1) 
    d7 <- data.frame(x = rep(c(-.5, .5), 20), cov = rnorm(40, sd = 1))
    d7$y <- .5 * d7$x + .6 * d7$cov + rnorm(40, sd = 1) 
d8 <- data.frame(x = rep(c(-.5, .5), 20), cov = rnorm(40, sd = 1))
d8$y <- .5 * d8$x + .6 * d8$cov + rnorm(40, sd = 1) 
    d9 <- data.frame(x = rep(c(-.5, .5), 20), cov = rnorm(40, sd = 1))
    d9$y <- .5 * d9$x + .6 * d9$cov + rnorm(40, sd = 1) 
d10 <- data.frame(x = rep(c(-.5, .5), 20), cov = rnorm(40, sd = 1))
d10$y <- .5 * d10$x + .6 * d10$cov + rnorm(40, sd = 1)
$\endgroup$

1 Answer 1

1
$\begingroup$

Some form of multivariate meta-analysis (also known as multi-level meta-analysis) would seem appropriate. You have a number of primary studies each giving rise to two outcomes which are known to be correlated. So you have all the necessary information to fit a model with a random effect for estimate and a random effect for study. You would also have a moderator (a two-level factor) for estimate type (adjusted versus not-adjusted). Since you seem to be using R I would suggest the following links Berkey et al example or Konstanopoulos et al example might well help. Or indeed any of the other analysis examples linked from those pages.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.