Evidence aggregation: combining estimates of a treatment variable from two regressions

Question

I'm working on a paper, which partly replicates some findings from a previous paper in my field. To be more particular, I am running the same model, estimating the same treatment variable, using exactly the same design, on new data in the same context.

The findings from my models differ somewhat from the study I am replicating, and I am now looking for a way to aggregate the evidence (i.e. the estimates of the treatment effect) across the two studies. I think the paper would benefit from a clear, formal statement about what we know now, based on the two studies.

In both cases, the treatment variable is estimated using least squares regression. My first idea was to aggregate the evidence by running a Bayesian linear model, using the findings from the first regression as a power prior for the treatment variable. The posterior would then reflect both the previous findings (in form of the prior) and the new evidence. A senior researcher in my field suggested that I should look into inverse-variance weighting, which I know is frequently used in meta-analysis. Of course, this is a form of meta-analysis, albeit with N = 2.

I know what to do to proceed with the Bayesian analysis, but would be happy to hear about any alternative ways to aggregate this evidence. Any suggestions on how to proceed with this are greatly appreciated.

Answer 1

You can indeed run a meta-analysis using standard software for that purpose. The only thing I would be cautious about is placing too much reliance on the estimate of heterogeneity $\tau^2$ (or $I^2$) that you get. With only 2 observations estimating a variance is going to be very imprecise. In turn this means that a random effects meta-analysis will be using an imprecise estimate of $\tau^2$. But as long as you are cautious I see no problem in that approach.