# Explain in layperson's terms why predictive models aren't causally interpretable

Imagine that you are asked to infer some causal effect -- a change in an outcome $$y$$ in response to some variable $$x$$. But, the person asking for this directs you to use a predictive model to do so. Here's the setup:

• $$x$$ is confounded inasmuch as there is some unobserved $$u$$ that is causally linked to both $$y$$ and $$x$$. We have a classical omitted variables bias.
• We have high dimensional covariates $$\mathbf{Z}$$ that are not independent of $$y$$ or $$x$$ and/or $$u$$
• You are asked to train a suite of predictive models -- neural networks, boosted trees, whatever -- denoted $$g_i([x, \mathbf{Z}]) + \epsilon$$ where $$i$$ indexes different models, and then select among them model $$i$$ that minimizes some metric of predictive skill. RMSE, for instance.
• Based on the chosen model, you are asked to report $$\frac{\partial \hat{y}}{\partial x} = \frac{\partial \hat{g}_i([x, \mathbf{Z}])}{\partial x}$$
• You know that $$E\left[\frac{\partial \hat{y}}{\partial x}\right] \neq \frac{\partial y}{\partial x}$$ in the population, because the error term includes the omitted variable, so therefore $$\frac{\partial \epsilon}{\partial x} \neq 0 \text{ in the population, despite the fact that } \frac{\partial \hat\epsilon}{\partial x} = 0$$ in any reasonable model $$g$$.

On top of omitted variables bias, there may be bias from regularization too!

• Further assume that you have some causal model -- say an instrumental variables regression, utilizing some suitable instrument $$w$$ for $$x$$. It's one of the models in your suite of models, but its predictive skill in terms of cross-validated RMSE is worse than the others.

The best model is the one that produces the consistent causal estimate, right? But:

How would you explain this to someone in layperson's terms?

The person asking for analysis doesn't understand causal inference, and needs to be educated. However, they don't understand math and have little attention span. How can you effectively convey the basic point that causal methods are required, and predictive methods are inappropriate? No math, lots of stories, pithy sentences.

• I wonder if the $\frac{\partial y}{\partial x}$ notation conveys the right message. Do you not need $P(y|\text{do}(x))$ or counterfactual notation for that? This is based on "T-consistency vs. P-consistency" (see the answer by Carlos), among other. Commented Dec 1, 2021 at 14:09
• to a layman the simplest explanation is to mention racial profiling, which should explain everything. if you build a predictive model with race as the sole X-variable and propensity to steal from your shop as Y-variable, you'll end up with a statistically significant model that probably has a good explanatory power in some countries. So, does this mean that race causes criminality? It doesn't. Precisely for some reasons that you mentioned, e.g. prevalence of poverty of people in one race can be a driver of theft occurrence. Commented Dec 1, 2021 at 14:12
• @Aksakal I would argue that racial profiling would be a poor choice of an example to unambiguously carry this point to laymen, as in a non-negligible proportion of people you will just try to argue rationally against well-established heuristics, stereotypes and assumptions. Better to introduce a non-charged example where you don't put your audience in a defensive state.
– Kuku
Commented Dec 1, 2021 at 15:44
• I think the main issue with the race thing is that "there is no causation without manipulation". One couldn't really design an experiment -- even hypothetically -- where we do a ceteris paribus race change and observe criminality. So it's an imperfect example for that reason. Commented Dec 1, 2021 at 16:08
• @Aksakal indeed, there is rationality from the predictive perspective, and that is what makes it an example where the orthogonality of these aspects is harder to carry through. Regarding the intersection of people, I think it we're sidetracking to an off-topic discussion, but I would not bet on such an assumption neither for laymen nor for academics, and it is an unnecessary risk for the task at hand (e.g. a recent counterexample in the trend of AI-neo-phrenological research: medium.com/@CoalitionForCriticalTechnology/…)
– Kuku
Commented Dec 1, 2021 at 16:09

First of all, I don't think this should be treated as a strict dichotomy: "predictive models can never establish causal inference." There are various situations in which a predictive model gives us "pretty darn good" confidence that a given causal relationship exists. So what I'd say is that predictive models - no matter how sophisticated - are often insufficient to establish causality with a high degree of confidence. Now, how to explain this to people who don't know stats/math at all?

Here's one approach:

You've heard it said that "correlation is not causation." What that means is just that just because two variables (call them A and B) are correlated, that doesn't mean one causes the other. This can happen when the correlation is due to a third "confounding" variable that is correlated with both A and B. For example: just because having a college degree is correlated with high earnings as an adult doesn't mean that getting a degree CAUSED those earnings to go up - it could be that "having rich parents" both allows people to get a degree and then separately helps them earn more (even if going to college actually does nothing).

Predictive models try to account for this problem by statistically "controlling for" confounding variables. So in the above case we could use statistical modeling to analyze the relationship between a degree and earnings after accounting for the fact that people with rich parents are more likely to have a degree.

Unfortunately, it's never possible in practice to control for EVERY confounding variable. This is partly because important variables (like the student's "personal motivation") may not exist or be impossible to measure. Even controlling for "parents being rich" is tricky - what single number could perfectly capture a family's entire economic situation? And how can we be sure that the data we have are accurate? Do any of us know PRECISELY how "rich" our parents were when we were growing up?

Another problem with predictive models is that even if you COULD control for everything they can't distinguish between A causing B or B causing A. So if we were trying to analyze the effect of depression on opiate use, no matter what control variables we include we can't be sure that the effect we observe isn't just due to opiate use CAUSING depression. Note that this is probably NOT a problem for our earlier example because it's impossible for your earnings as an adult to CAUSE you to have gone to college earlier in your life. This is one way in which our theoretical understanding of how these variables work helps us to understand the threats to causal inference.

The only way to completely ensure that a relationship between A and B is causal is to experimentally control how people get "assigned" to different values of A (e.g. to get a college degree or not). If assignment to A is completely random then we can be sure that NOTHING else influenced A, which means that you don't have to worry about ANY confounding variables (even B) in analyzing the relationship between A and B. However, for reasons that are obvious when we're considering college degrees, random assignment is often infeasible or downright unethical. So we have to use other research design approaches to approximate the causal power of random assignment. Critically, these other approaches (instrumental variables, regression discontinuity, natural experiments) rely on the features of the world itself, and the data collection process, rather than statistical/mathematical methods, to address issues of confounding variables.

• Nice, clear answer. Maybe the following is too technical, but I would also argue that including too many variables hampers causal inference as well. For example, including post-treatment variables, even if the treatment effect is otherwise unconfounded, will mess up the effect estimate. This is separate from yet another issue, which is conditioning on colliders. I only mention this because a possible response from the person challenging OP is "what if we control for a whole lot of variables?" They might be enough to reduce confounding, but they could induce bias as well!
– Noah
Commented Dec 1, 2021 at 20:08
• I like that approach. I also like to point people to Tyler Vigen's excellent page of spurious correlations, both the show the possibility of false positives in unexpected places, and as a cautionary tale: you stand a chance of looking very silly if you read too much into correlation. tylervigen.com/spurious-correlations Commented Dec 3, 2021 at 21:57

I think this explanation is best approached sequentially. Start with a simple story:

When my dog Omey wags his tail, that indicates he is happy. For instance, he never wags it at the vet, wags it a bit when I get his leash, and wags a whole lot when I also grab a tennis ball. But if I wag Winston's tail for him, it usually has the opposite effect.

In other words, a "wagging tail" is a good predictor of my dog's state of mind, but I cannot use this knowledge to make him happy (except as a kind of proxy variable in experiments). Here the causality is pretty straightforward, so the contrast between prediction and cause is stark.

The next parable is both more realistic and closer to home:

If you look at the performance of salespeople at my company, the ones with expensive cars are the most productive. While it is possible that clients find luxury cars impressive, and that makes selling to them easier, our sales happen over the phone, so it is unlikely that giving our salespeople nice cars will boost revenue (unless there is a promise to let the customer take the Porsche for a spin after the deal is sealed).

The causality goes the other way here, though there is a slight potential for the correlation between sales and cars to be causal.

Now for another example:

It is evident that people who have our app installed on their phone buy more than folks that only shop in person and/or through the website. The app sends notifications and it makes it much easier to buy stuff with just a click. But people don't just install the app for no reason. They do it because they expect to buy more, which the app makes more convenient, so comparing customers with and without the app is like comparing apples and orangutans. They are very different people.

Here there is causality in both directions, but arguably the high intent $$\rightarrow$$ expenditures mechanism dominates the app install $$\rightarrow$$ expenditures. When a causal explanation works in both directions, you can usually settle the debate with an experiment to see which one is the most important.

The real world is vastly more complicated than these fairly simple stories, and our intuition can often lead us astray at a great cost. Here are two more good examples from industry of mistaking correlation for causation:

1. Ascarza, Eva. Retention Futility: Targeting High-Risk Customers Might Be Ineffective. Journal of Marketing Research (JMR) 55, no. 1 (February 2018): 80–98.
2. Blake, T., Nosko, C. and Tadelis, S. (2015), Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment. Econometrica, 83: 155-174. https://doi.org/10.3982/ECTA12423
• I really like this. It starts with a very simple/accessible case that makes the main point, and then adds in a little bit at a time in later examples. If someone only understands the first example, that's still progress. And if you can get them to go further, all the better. Commented Dec 3, 2021 at 14:39

I don't think you even need to posit a covariate adjustment set $$\textbf{Z}$$ nor the indexation of black-box models to convey in layman terms the main point. Assume the following:

• $$y$$ is number of people drowning in a given month in a given city
• $$x$$ is number of ice-cream sold in a given month in a given city
• $$u$$ is temperature in a given month in a given city, the unobserved confounder

$$x$$ will be highly predictive of $$y$$, and most likely a model just using $$x$$ as a predictor will outperform models that use noisier measurements of the real causes or their instrumental variables.

Clearly, the best predictive model is not necessarily the one that gives the most consistent causal estimate.

• Ice cream and drowning is a good example. The difficulty with the high-dmensional covariate adjustment set is that multicollinearity can further muddy a causal model. Not by adding bias, but rather by adding variance. Commented Dec 1, 2021 at 15:44
• I am not sure I follow how that would alter the point at hand. It would just introduce an arbitrary preference for predictors that are measured in the most independent and precise manner (which, again, have no reason to be the causal ones).
– Kuku
Commented Dec 1, 2021 at 15:47
• Your point is still valid. The reason why it's relevant to the totality of my problem is that the best causal estimate might be the one without all of those predictors, even though those predictors might lower MSE. Commented Dec 1, 2021 at 15:53
• Then a secondary problem is making sure that the person asking for analysis is very clear on what their goal is for analysis. Sounds like the first step is to make sure that the person asking for analysis has a clear, explicit goal of what he wants the results to be used for: to understand/explain part of the phenomenon or just to predict new instances? Once that is clear, you could motivate with such an example that minimizing prediction squared error and getting a precisa causal estimate are clearly different things. I am not sure though this is something to expand in this answer.
– Kuku
Commented Dec 1, 2021 at 15:59
• The problem here is that you cannot estimate a parameter like the average treatment effect, and obtain confidence intervals, from a predictive model unless it's incorporated into a causal inference model (like targeted maximum likelihood estimation or double machine learning), or if you use a simple parametric $g$-computation model like least squares regression. But even before you start modeling, a causal diagram should be built in order to identify unwanted variables that ought to be excluded from the model. Commented Dec 1, 2021 at 17:14

Correlation does not equal causation. Predictive models using advanced techniques such as machine learning can be quite good at finding associations between predictive variables and an outcome, but this isn't the same as determining the causal relationships between those variables.

For example, as a researcher you may find a strong correlation between homelessness ($$Y$$) and illegal drug use ($$X$$) in a city, and could even state with a high degree of accuracy you can predict a person is homeless if you know they're a drug user. Can you confidently report to the city government that illegal drug use causes homelessness: $$X \rightarrow Y$$, and therefore reducing drug use will reduce homelessness?

No, not without either deducing or collecting more information about the causal relationship between $$X$$ and $$Y$$. Perhaps it's the reverse, homelessness causes a higher risk for illegal drug use: $$X \leftarrow Y$$? Or perhaps $$X$$ and $$Y$$ are not as closely related, or even completely independent, and there's in fact a third variable such as mental illness ($$Z$$) that causes both homelessness and illegal drug use: $$X \leftarrow Z \rightarrow Y$$? In both of these cases the structure of your causal inference model will have to be altered from what you might see in a typical predictive model. There are many other possibilities as well (such as mediator and collider variables) which must be accounted for or ruled out in order to draw a complete picture of the cause and effect relationships.

• this "correlation causation" thing irks me every time I hear it. I'm afraid today this phrase should be left to politicians to mutter, because it's over/mis-used too much. it also is too coarse, lacks appreciation of nuances. e.g. there's a philosophical view that correlation is all we should care about in science, because that is the only thing we can truly establish. finally, it is also tautological in the context of the question, because that is precisely what OP is asking: explain why correlation is not causation. think of this: predictive model at best would determine correlations Commented Dec 1, 2021 at 14:54
• @Aksakal The OP wanted a pithy sentence - maybe "prediction does not equal causation" would be better. But agreed you need more than a pithy sentence to explain the concepts here. Commented Dec 1, 2021 at 15:01
• @RichardHardy Thanks for spotting that typo - now corrected! Commented Dec 1, 2021 at 15:02
• if "correlation causation" became some sort of established colloquial fact in contemporary Anglo-Saxon society due to its frequent use then maybe you're right, maybe to a layman the best answer is to equate OP's question to this phrase, and that would do it Commented Dec 1, 2021 at 15:06

Oo Oo! I'm a mathematical layperson! Let's see if I can do this:

TLDR: I use predictions (or "predictive models") to prepare for events beyond my control without having to know what actually causes them.

I might posit that a lay predictive model is "whether the weather report says it will rain this weekend". I may not care what will cause it to rain or not rain on this particular weekend, I cannot change it, and I only care about whether to pack my fishing equipment.

Contrast this with something I want to control: The weather in my house!!!

1. If I keep my roof in good shape, it probably will not rain.
2. If I set the thermostat for 74 degrees, it will probably stay between 72 and 76 degrees.
3. If I keep my windows closed, it will probably not be windy.

Etc.

I can depend on known, causal chains to control some things. But, I use predictive models to prepare for things beyond my control.

• Pretty good! If you whack the weatherman, it will still rain. Still, this doesn't quite get at the real problem. You never ask the weatherman how weather would change if you could magically adjust air pressure for a while at some place. Commented Dec 2, 2021 at 20:48
• This appears to be a false dichotomy. Many causal models are built exactly with the aim of predicting what will happen if you do $x$. E.g. you want to predict what will happen with your pain if you take some painkillers; or what will happen to your turnover and profit if you decrease the price of a good you are selling; or what will happen to unemployment if you (being the central bank) raise interest rates; etc, etc. Few causal models are used for anything else than prediction (historical analysis is one thing I can think of). Commented Dec 2, 2021 at 21:43

The basic problem is that a non-causal predictive model may fail if used for interventions. Start with a very simple example where an important part of the real world has:

$$Cause \rightarrow \mathit{Effect} \rightarrow Measurement$$

Because we imagine Measurement is nearly perfect, the best predictive model will prefer it to a noisier Cause, but Measurement can only be known after Effect, so it cannot inform policies or interventions. Changing Measurement by some other means won't change Effect at all.

 ~ * ~ Possibly that's sufficient, but more detail below. ~ * ~


If Measurement is something like income or health or quality, you can of course measure it ahead of time, but then it's not really the right measurement, because it hasn't taken Effect into account yet. The predictions won't be as good. Here you would have done better to use Cause, if known.

Often Cause is not known, and Measurement could be as good a predictor as you like. In the ultimate example, we have entangled particles:

$$\mathit{Spin\ 1} \leftarrow \mathit{Hidden\ Cause} \rightarrow \mathit{Spin\ 2}$$

In this case, Spin 2 is a perfect predictor of Spin 1, but it can't ever be used to change Spin 1: they both simply reflect a hidden common cause. In a less extreme example, ice-cream sales and drownings are both caused by heat, but also other things:

$$Other1 \rightarrow Sales \leftarrow \mathbf{Heat} \rightarrow Drownings \leftarrow Other2$$

We can't reduce drownings by banning ice-cream, and we can't do much about Heat, however predictive, but we can make policies about Other2. (And ice-cream sales could be an early warning allowing us to act.)

Usually there is no one perfect Measurement, so we manufacture one: our predictive model cleverly combines Many things. But even if it correctly accounts for dependencies among Many things, it doesn't help policy or interventions if Many things are downstream of Effect.

If Measurement is downstream of both Cause and Effect, then the model can fail in stranger ways, esp. if it was cleverly trying to control things statistically.

If it's a person who really knows barely anything about statistics and causation I would provide some examples that are straightforward.

If you have data on the number of bathrooms in someone's largest house, you can classify the person as a billionaire or not, in the sense that billionaire mansions have numerous bathrooms. I'd assume you could achieve some precise estimates in this problem. However, increasing the number of bathrooms in someone's house won't make them billionaires. So you could use such a model to identify billionaires, but not to make someone a billionaire. Correlation, but not causation.

Another common example is the ice cream consumption relationship to shark attacks (being confounded by temperature). In some places, you could get some reasonable prediction of shark attacks based on ice cream consumption. But throwing more sharks in beaches wouldn't make people eat more ice cream (at least not to the same extent you measured with the confounded estimate).

I agree with Graham Wright that this should not be a dichotomy. You can have predictions based on causal models, for example. If the person knows some statistics, I would go for a different approach, like the other answers, but if it's my not-even-high-school-diploma grandma, I would go for the intuitive examples I gave above.