# Meta-Analysis on Effect Sizes with 95% Bayesian CI from CausalImpact R package

I am using the CausalImpact package in R to calculate the impact of a marketing intervention using Bayesian Structural Time Series. This methodology and package is explained in Broderson et al. 2015 found at https://research.google.com/pubs/pub41854.html (direct PDF link here).

I calculated the impact of a marketing intervention in 18 clinics (the "test" subjects). Using the package, I calculated the causal impact of the intervention compared with the respective synthetic control in each of the 18 clinics.

A replicable example of one sample intervention is available from Google's Github page on CausalImpact and the code is provided below:

library(CausalImpact)

#code from Google's Github page on CausalImpact package
set.seed(1)
x1 <- 100 + arima.sim(model = list(ar = 0.999), n = 100)
y <- 1.2 * x1 + rnorm(100)
y[71:100] <- y[71:100] + 10
data <- cbind(y, x1)

pre.period <- c(1, 70)
post.period <- c(71, 100)
impact <- CausalImpact(data, pre.period, post.period)
summary(impact)


For this example, one can pull out the "Relative Effect" (effect size), along with the lower 95% CI, upper 95% CI, and SD values for this one example intervention.

impact$summary$RelEffect[1]
impact$summary$RelEffect.lower[1]
impact$summary$RelEffect.upper[1]
impact$summary$RelEffect.sd[1]


I have done this for 18 different "test" clinics. For each one, I have the effect size along with the corresponding 95% CI and SD.

My question is this: In what way can I summarize the intervention in one single, summary metric while taking into account variance for each result?

I believe this involves a random effects Meta-Analysis approach by inverse weighing of the variance, but I am not certain. I tried looking into the Metafor package in R (http://www.metafor-project.org/doku.php/analyses), but I cannot seem to find an appropriate analysis or code. Most of the examples I have seen require a sample size for multiple studies. The best I can think of is that my results are analogous to 18 different studies with a sample size of n=1, though I do not think that is a valid interpretation. As the CausalImpact methodology is based on a Bayesian approach, the CI are also not necessarily symmetrical (as seen in the example above). I am also uncertain how to present these results in an appropriate forest plot.

Any help on getting one summary metric is hugely appreciated. I apologize for any errors I may have made. Thank you.

• Would one of the bayesain packages mentioned in the CRAN Task View help? CRAN.R-project.org/view=MetaAnalysis – mdewey Oct 25 '17 at 12:51
• I am looking into them. Unfortunately, I have not had luck so far, but will update if I find the correct approach. I appreciate you bringing them to my attention. – user181973 Oct 25 '17 at 16:00
• Is it feasible for you to present the output - Broderson et al (2015) ? And what prompted you to know standard error for your study ? and probably you do not have sample-sizes of studies ( or clinics). – Subhash C. Davar Nov 15 '17 at 4:08

The general problem you describe is covered in the paper available here: Extending Bayesian structural time series.... They perform an analysis similar to yours but for an econometrics topic where they want to estimate the effect of a conservation program on water savings in California. To obtain the weights for the weighted meta-model in metafor they used the a tranform of the credible intervals from the causalimpact step.