Conduct meta-regression in R {metafor} with multiple time-points

Question

I need to conduct a meta-analysis for a publication, but this is my first meta-analysis and I still don’t feel confident. I will describe the steps I have followed, and hopefully some of you might find errors on my methods, and suggest alternatives.

For this particular analysis, long-term studies should be more important than short-term because a few experiments showed transient effects, hence short-term studies might fail to capture that the effect is not really significant in the long-term. On my dataset, about half the studies have several non-independent measurements taken at different time-points (i.e. several annual measurements). Other experiments, despite having been carried out for several years, show the data already aggregated, with only one row per study with mean and standard deviation. I considered running a multivariate meta-analysis to solve this issue, but it would unbalance the analysis, giving more importance to the experiments with several rows of data (annual measurements) than to the experiments with aggregated data in only one row (pooled across several years). Am I right? This is an example of the dataset:

• Study 1, Year 1, Effect Size 1
• Study 1, Year 2, Effect Size 2
• Study 1, Year 3, Effect Size 3
• Study 2, Year 1, Effect Size 4
• Study 3, Years 1-4, Effect Size 5
• Study 4, Years 1-3, Effect Size 6

Alternatively, I decided to try and aggregate the data, so that finally there is only one row per study. I followed these steps:

Calculate effect sites for each row, including those studies with several rows (annual data). In this case, I calculated the log response ratio (ROM):

dat <- escalc (measure="ROM”, n1i=elev.rep, n2i=control.rep, m1i=elev.ANPP.mean, m2i=control.ANPP.mean, sd1i=elev.SD, sd2i=control.SD, data=all)

Aggregate studies using the function agg {MAd}. I used the Borenstein et al. 2009 method, and correlation=1:

datAgg <- agg(id = id,es = yi,var = vi, cor =1,method = "BHHR", data = dat)

I have now only one row per study. However, since long-term experiments are more important, I have created user-defined weights that take into account the number of replicates and the number of years of each study:

datAgg$weightsTime <- with(datAgg, ((control.rep * elev.rep)/(control.rep + elev.rep)) + ((nyears^2)/(2*nyears)))

Run the mixed-effects meta-regression with two moderators, using Hedges Estimator (HE) and the Knapp and Hartung approach:

m <- rma.uni(yi, vi, mods= ~ factor(A) * factor(B), method="HE", data=datAgg, weights=weightsTime, knha=TRUE)

Am I doing something wrong? Can this method be improved? So far the results confirm my hypothesis, but of course I might be using a sub-optimal approach. Many thanks

Answer 1

I agree that 'multivariate meta-analysis' in this case is hard to justify, particularly if there is a possibility of repeated measures effect (however, in that case, why would it be reported on an annual basis, was there any indication that the samples may have been independent? this would be insightful to explore). In particular, rma.uni has an assumption of 'independent studies'.

Based on your justification, it seems reasonable to use weights for the length of the study (although it would help if you justify why you use those particular weights). Yet, an alternative approach, if your sample size allows, would be to do separate analyses for those two categories (log and short-term) to avoid information loss.

Weights that take into account the number of replicates are open to debate, however, because if a study had sufficient power, it is not entirely defensible to down-weigh it due to lower sample size (the model is not accounting for within-study variability). Also note that, unless you have very long studies (unlikely) or very few reps, your reps penalty is worse/stronger than the length of study penalty, which I think is unfortunate.

The agg part seems well-supported. rma.uni has the same assumptions as GLM, so if satisfied, it should be ok.