3
$\begingroup$

Say I am trying to predict depression from anxiety. I collect data and build a MLE and obtain r=0.9. To me, this is great, so I push the model to production. 4 months later, I realise that the "rate of unemployment" is a confounder that plays on both variables.
I conclude that I should not merely look at correlation, but rather tackle predictive analysis from a causal inference perspective.
However, when learning about predictive analysis, I rarely see mention of causal inference. It is often just about "finding the best model to fit your model".

$\endgroup$
6
  • $\begingroup$ Welcome to Cross Validated! I'm not sure I completely understand your question. You observe a correlation between X and Y, but realize that Z is a confounding variable. You, presumably, realized that Z is confounding through correlational analysis (note: it's possible that there's yet another variable or set of variables that explains the relationship between rate of unemployment and your measurements). I mean to say that statistically finding a confounding var. doesn't mean the confounding var. is "causal" either (there could be deeper explanatory variables), though this is near philosophy. $\endgroup$ Commented Jan 24, 2022 at 19:49
  • 1
    $\begingroup$ I think the answer is "it depends". It depends on what exactly you're trying to do. So, could you describe the specific problem you're trying to solve? Or are you asking generally? $\endgroup$ Commented Jan 24, 2022 at 19:52
  • $\begingroup$ the confounder may show that during periods of low unemployment, perhaps there is no correlation at all between depression and anxiety, and with the confounder included, we may be able to predict the dependent variable much better. the r-value doesn't represent the true correlation between depression and anxiety. @VladimirBelik $\endgroup$
    – Anon
    Commented Jan 24, 2022 at 20:18
  • $\begingroup$ I totally agree! That's a fantastic example. But again - what exactly is your question? $\endgroup$ Commented Jan 24, 2022 at 20:20
  • $\begingroup$ @VladimirBelik simply because X correlates to Y for a set of observations doesn't mean that X is a good predictor of Y in the absence of a confounding variable. Perhaps X doesn't show any correlation to Y when the confounder is conditioned upon. My question is whether this holds true? $\endgroup$
    – Anon
    Commented Jan 24, 2022 at 20:25

2 Answers 2

3
$\begingroup$

As Vladimir mentions, the answer is "it depends". If you build a well-calibrated correlational (i.e., predictive) model on units randomly sampled from your population, then that model should generalize to other members of the population. If you have a model of depression and anxiety, then applying that model to other units in the population will be accurate. The fact that there are confounding variables says nothing about the validity or generalizability of the model results (to the same population).

For example, if you calibrate a model well to predict depression participants' reported anxiety, the fact that there are confounders of that relationship doesn't affect the quality of the prediction when applied to new data. Of course, the presence of a confounder implies that there is a variable predictive of the outcome that you are missing, in which case your predictions would be improved if you included it in your model (the same as any prognostic variable).

Where you need causal inference is when you want to predict outcomes after (hypothetically) manipulating a feature. For example, if you manipulate participants' anxiety by giving them an anti-anxiety medication, your previous model for depression is no longer valid because the outcome and feature are confounded in the original data. Your model would need to account for all confounding factors and not condition on any confounding-inducing variables to be able to predict the outcome without bias. You need a statistical theory of causal inference to tell you which features to include and exclude in your model to arrive at an unbiased model for the outcome given manipulation of the treatment.

$\endgroup$
3
  • $\begingroup$ In the example I gave above, since X highly correlates with Y, we can say that X is likely to cause Y, however in the case where there is spurious correlation, if some confounder is conditioned upon, then there could be no correlation at all. e.g., let's say we observe that ice cream sales is causally correlated with drowning rates (r=.5), if a confounder is the weather, and we look at the correlation when weather is hot, then there may be no correlation at all. Furthermore, with this confounder accounted for, one can better predict the drowning rate. $\endgroup$
    – Anon
    Commented Jan 24, 2022 at 20:32
  • $\begingroup$ If you want to predict ice cream sales, drowning is a good predictor even though there is confounding. The problem is when you try to increase drowning to increase ice cream sales, or when you try to generalize the finding to a population other than the one from which your training data was sampled. Of course including weather in the model would be a better model, but the drowning-only model is still valid and will be effective at prediction despite the lack of causal relationship. $\endgroup$
    – Noah
    Commented Jan 24, 2022 at 20:42
  • $\begingroup$ thank you. thank you. $\endgroup$
    – Anon
    Commented Jan 24, 2022 at 20:52
1
$\begingroup$

First of all, you shoud have in mind that the estimation of causal effects and prediction are not the same thing. The amount of consumption of chocolate in a country might help you to predict from which country the next Nobel laureate in medicine will come from, but it will not tell you that much about the causal "story" that produces nobel laureates in some countries (probably that story has something to do with economic growth).

To your question:

Yes, if ...

...you want to generalize your results to new environments, $e$ . Causal relationships of the kind $X^{e} \rightarrow Y^{e}$ should go along with so called invariant conditional distributions $P(Y^{e} |X^{e})$, which are the same for all environments $e\in \mathcal{E}$ from some environment space $\mathcal{E}$. So having $X^{e}$ in your model when predicting $Y^{e}$ should improve the robustness of your predictions when dealing with different environments. An example from this book Ref1: let $X^{e}$ be the height of a mountain and $Y^{e}$ the temperature measured at a certain height. So you might deduce from some physical theory that there is something like a causal relation between height and temperature ("the higher, the colder"). This relationship should hold between different environments, such as the Swiss and the Austrian Alps. Hence your prediction model built on data from the Swiss Alps should also hold for the Austrian Alps (robust prediction). See this great article here: Ref2. Or this nice talk: Ref3.

No, if ...

... you are just interested in the predictive performance of your model based on your training data (one environment).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.