I'm new to CausalImpact package in R. I'm trying to understand what the p-value or Posterior tail-area probability mean in the summary. Does it mean that because the p-value is very low, the chance that there's an effect of an intervention is significant? How to put it better in layman's term?
Your understanding is correct: if the tail-area probability is small, the effect of the intervention can be considered significant. Technically, this value is calculated as follows:
- Based on parameter priors and the data you provide, CausalImpact calculates the posterior distribution of the response variable that would be expected in the absence of an intervention.
- The actual response is then compared to this posterior distribution. The tail-area probability is the probability under the calculated posterior that the response is at least as extreme (away from the expected value) as the observed one.
In the example (to be run with
example("CausalImpact") in R), the summary shows a positive effect and reports a tail-area probability of 0.001. This means: if the intervention had no effect on the response variable, there would be a chance of only 0.1% to see a positive effect at least as large as the one observed.