2
$\begingroup$

Short version: How useful are predictions of a variable y that are obtained using theoretically unrelated variables X that happen by mere luck to predict y very well? Is there any paper out there dealing with this issue?

Long version: For example, let's say that I have time series data on y (monthly crime rates in Belize), and I need to estimate the causal impact of an intervention following the methodology in Brodersen et. al. (2015). To do this, I'd have to estimate a model that predicts my data very well for the period previous to the intervention, and use the predictions for the post-intervention period as a counterfactual scenario. Then the difference between expected and observed y would be a measure of the impact of the intervention. For inferring causality, I'd need a counterfactual created with variables that were not affected by the intervention, but that create a good predictor of the series without the intervention.

To do this, I employ series that are highly correlated with y, but that are certainly unrelated from a theoretical point of view (for example, monthly number of Google queries for "sushi" from Burkina Faso extracted from Google Correlate). These should satisfy the assumptions mentioned above. However, the guarantee that comes from a causal relationship is just not there. Does this mean that I can't measure the causal impact of interest using that counterfactual?

$\endgroup$

1 Answer 1

1
$\begingroup$

For inferring causality, I'd need a counterfactual created with variables that were not affected by the intervention, but that create a good predictor of the series without the intervention.

(Emphasis added.)

This is where your proposed method breaks down. The monthly number of Google queries for "sushi" from Burkina Faso may not be affected by the intervention, but there is no reason to expect that they would be a good predictor of the series without the intervention.

To model and predict your counterfactual, you will need predictors that actually have predictive value, based at least on theory.

$\endgroup$
3
  • 1
    $\begingroup$ Thank you for your answer. I still don't see why a spurious predictor does not have predictive value in this case. Imagine for example that X has a high (but spurious) correlation with y previous to the intervention, and that the intervention happens leaving everything else constant. In that case, because X is unaffected by the intervention, wouldn't it remain a good spurious predictor of y in the absence of the intervention? Wouldn't that be enough to estimate a counterfactual? $\endgroup$ Feb 7, 2018 at 8:52
  • 1
    $\begingroup$ If $X$ is spurious, then it will not help predicting $y$ in the absence (or the presence) of the intervention. So why would you include it in your model? $\endgroup$ Feb 7, 2018 at 8:55
  • 1
    $\begingroup$ that's a good point, and I might be missing something very basic, but is causality a necessary condition for prediction? Conversely, does spurious correlation disqualify a covariate from being a good predictor of y? $\endgroup$ Feb 7, 2018 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.