Meta-analysis in R of pre-post treatment design studies

Question

I would like to conduct a meta-analysis in R of studies using a pre-post treatment design. In every individual study 4 different subgroups of patients have undergone a pharmacological treatment and their symptoms are measured on a continuous scale. For every study mean and sd of the effect size (cohens d) for the treatment effect is available for every group (this makes the analysis for complicated then usual meta-analysis in which only a single outcome measure per study is used).

What is an appropriate way to investigate the effect of treatment (difference between pre- and post-measurement) as well as the effect of group as well as the group-treatment interaction? Would the below example be a easy solution?

Here is some R code for dummy data:

set.seed(123)
a <- c(rep("study1",4), rep("study2",4), rep("study3",4), rep("study4",4),       rep("study5",4), rep("study6",4))
b <- rep(c("group1", "group2", "group3", "group4"),6)
data <- as.data.frame(cbind(a,b))
names(data) <- c("study", "group")
data$d <- rep(c(1,2,3,4),6) + rnorm(24,0,0.5)
    data$sd <- rep(1, 24) + rnorm(24,0,0.25)


library(metafor)

model_1 <- rma(d, sd, data=data)
summary(model_1)
forest(model_1)

model_2 <- rma(d, sd, mods=group, data=data)
summary(model_2)
forest(model_2)

Answer 1

Yes and no. First, a few corrections:

1) The second argument in the rma() function is for the sampling variances. However, you are specifying the SDs (of the d values) as the second argument. So, either use rma(d, SD^2, ...) or use rma(d, sei=SD, ...).

2) For the second model, you should use model_2 <- rma(d, sd, mods=~factor(group), data=data).

3) Although it will work in this example, I would suggest never using data as the name of an object, as data is actually a function.

Now back to your example. For the particular data that you have simulated, this approach would be okay. However, try changing the data$d line to:

data$d  <- rep(c(1,2,3,4),6) + rep(rnorm(6,0,2), each=4) + rnorm(24,0,0.5)

So, what I have introduced is a shift of all effects within a particular study by some constant amount. The data are still simulated without heterogeneity (i.e., the true difference between groups is still constant across studies; e.g., group2-group1 = 1 in all studies). However, due to the specific circumstances under which a particular study is conducted, all d values are shifted by a constant amount. A look at forest(model_2) will show you how the estimated effects are now always either too large or small for these data.

Now try this:

model_3 <- rma(d, sd^2, mods = ~ factor(study) + factor(group), data=data, btt=7:9)
summary(model_3)
forest(model_3)

Now this model includes fixed study effects. A look at forest(model_3) shows that the fit is much better. In addition, you are directly getting the group contrasts from the model (the intercept corresponds to group1, so the coefficient for group2 is the contrast between group2 and group1 -- which is, not surprisingly, close to 1).

Also, by using btt=7:9, the omnibus test of moderators now only includes the three group dummies (which are automatically created when you use factor(group), so this is a 3 degrees of freedom test whether the group factor is actually significant.

If there is any additional variability that is not accounted for by the fixed study effects and the group factor (which in turn implies that the true differences between group2 and group1, group3 and group1, and so on, are not constant across studies), then this will be reflected in the estimate of tau^2.