Explain in layperson's terms why predictive models aren't causally interpretable

Question

Imagine that you are asked to infer some causal effect -- a change in an outcome $y$ in response to some variable $x$. But, the person asking for this directs you to use a predictive model to do so. Here's the setup:

• $x$ is confounded inasmuch as there is some unobserved $u$ that is causally linked to both $y$ and $x$. We have a classical omitted variables bias.
• We have high dimensional covariates $\mathbf{Z}$ that are not independent of $y$ or $x$ and/or $u$
• You are asked to train a suite of predictive models -- neural networks, boosted trees, whatever -- denoted $g_i([x, \mathbf{Z}]) + \epsilon$ where $i$ indexes different models, and then select among them model $i$ that minimizes some metric of predictive skill. RMSE, for instance.
• Based on the chosen model, you are asked to report $$ \frac{\partial \hat{y}}{\partial x} = \frac{\partial \hat{g}_i([x, \mathbf{Z}])}{\partial x} $$
• You know that $$ E\left[\frac{\partial \hat{y}}{\partial x}\right] \neq \frac{\partial y}{\partial x} $$ in the population, because the error term includes the omitted variable, so therefore $$ \frac{\partial \epsilon}{\partial x} \neq 0 \text{ in the population, despite the fact that } \frac{\partial \hat\epsilon}{\partial x} = 0 $$ in any reasonable model $g$.

On top of omitted variables bias, there may be bias from regularization too!

• Further assume that you have some causal model -- say an instrumental variables regression, utilizing some suitable instrument $w$ for $x$. It's one of the models in your suite of models, but its predictive skill in terms of cross-validated RMSE is worse than the others.

The best model is the one that produces the consistent causal estimate, right? But:

How would you explain this to someone in layperson's terms?

The person asking for analysis doesn't understand causal inference, and needs to be educated. However, they don't understand math and have little attention span. How can you effectively convey the basic point that causal methods are required, and predictive methods are inappropriate? No math, lots of stories, pithy sentences.

Answer 1

First of all, I don't think this should be treated as a strict dichotomy: "predictive models can never establish causal inference." There are various situations in which a predictive model gives us "pretty darn good" confidence that a given causal relationship exists. So what I'd say is that predictive models - no matter how sophisticated - are often insufficient to establish causality with a high degree of confidence. Now, how to explain this to people who don't know stats/math at all?

Here's one approach:

You've heard it said that "correlation is not causation." What that means is just that just because two variables (call them A and B) are correlated, that doesn't mean one causes the other. This can happen when the correlation is due to a third "confounding" variable that is correlated with both A and B. For example: just because having a college degree is correlated with high earnings as an adult doesn't mean that getting a degree CAUSED those earnings to go up - it could be that "having rich parents" both allows people to get a degree and then separately helps them earn more (even if going to college actually does nothing).

Predictive models try to account for this problem by statistically "controlling for" confounding variables. So in the above case we could use statistical modeling to analyze the relationship between a degree and earnings after accounting for the fact that people with rich parents are more likely to have a degree.

Unfortunately, it's never possible in practice to control for EVERY confounding variable. This is partly because important variables (like the student's "personal motivation") may not exist or be impossible to measure. Even controlling for "parents being rich" is tricky - what single number could perfectly capture a family's entire economic situation? And how can we be sure that the data we have are accurate? Do any of us know PRECISELY how "rich" our parents were when we were growing up?

Another problem with predictive models is that even if you COULD control for everything they can't distinguish between A causing B or B causing A. So if we were trying to analyze the effect of depression on opiate use, no matter what control variables we include we can't be sure that the effect we observe isn't just due to opiate use CAUSING depression. Note that this is probably NOT a problem for our earlier example because it's impossible for your earnings as an adult to CAUSE you to have gone to college earlier in your life. This is one way in which our theoretical understanding of how these variables work helps us to understand the threats to causal inference.

The only way to completely ensure that a relationship between A and B is causal is to experimentally control how people get "assigned" to different values of A (e.g. to get a college degree or not). If assignment to A is completely random then we can be sure that NOTHING else influenced A, which means that you don't have to worry about ANY confounding variables (even B) in analyzing the relationship between A and B. However, for reasons that are obvious when we're considering college degrees, random assignment is often infeasible or downright unethical. So we have to use other research design approaches to approximate the causal power of random assignment. Critically, these other approaches (instrumental variables, regression discontinuity, natural experiments) rely on the features of the world itself, and the data collection process, rather than statistical/mathematical methods, to address issues of confounding variables.

Answer 2

I think this explanation is best approached sequentially. Start with a simple story:

When my dog Winston wags his tail, that indicates he is happy. For instance, he never wags it at the vet, wags it a bit when I get his leash, and wags a whole lot when I also grab a tennis ball. But if I wag Winston's tail for him, it usually has the opposite effect.

In other words, a "wagging tail" is a good predictor of my dog's state of mind, but I cannot use this knowledge to make him happy (except as a kind of proxy variable in experiments). Here the causality is pretty straightforward, so the contrast between prediction and cause is stark.

The next parable is both more realistic and closer to home:

If you look at the performance of salespeople at my company, the ones with expensive cars are the most productive. While it is possible that clients find luxury cars impressive, and that makes selling to them easier, our sales happen over the phone, so it is unlikely that giving our salespeople nice cars will boost revenue (unless there is a promise to let the customer take the Porsche for a spin after the deal is sealed).

The causality goes the other way here, though there is a slight potential for the correlation between sales and cars to be causal.

Now for another example:

It is evident that people who have our app installed on their phone buy more than folks that only shop in person and/or through the website. The app sends notifications and it makes it much easier to buy stuff with just a click. But people don't just install the app for no reason. They do it because they expect to buy more, which the app makes more convenient, so comparing customers with and without the app is like comparing apples and orangutans. They are very different people.

Here there is causality in both directions, but arguably the high intent $ \rightarrow$ expenditures mechanism dominates the app install $\rightarrow$ expenditures. When a causal explanation works in both directions, you can usually settle the debate with an experiment to see which one is the most important.

The real world is vastly more complicated than these fairly simple stories, and our intuition can often lead us astray at a great cost. Here are two more good examples from industry of mistaking correlation for causation:

1. Ascarza, Eva. Retention Futility: Targeting High-Risk Customers Might Be Ineffective. Journal of Marketing Research (JMR) 55, no. 1 (February 2018): 80–98.
2. Blake, T., Nosko, C. and Tadelis, S. (2015), Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment. Econometrica, 83: 155-174. https://doi.org/10.3982/ECTA12423

Answer 3

I don't think you even need to posit a covariate adjustment set $\textbf{Z}$ nor the indexation of black-box models to convey in layman terms the main point. Assume the following:

• $y$ is number of people drowning in a given month in a given city
• $x$ is number of ice-cream sold in a given month in a given city
• $u$ is temperature in a given month in a given city, the unobserved confounder

$x$ will be highly predictive of $y$, and most likely a model just using $x$ as a predictor will outperform models that use noisier measurements of the real causes or their instrumental variables.

Clearly, the best predictive model is not necessarily the one that gives the most consistent causal estimate.

Answer 4

Correlation does not equal causation. Predictive models using advanced techniques such as machine learning can be quite good at finding associations between predictive variables and an outcome, but this isn't the same as determining the causal relationships between those variables.

For example, as a researcher you may find a strong correlation between homelessness ($Y$) and illegal drug use ($X$) in a city, and could even state with a high degree of accuracy you can predict a person is homeless if you know they're a drug user. Can you confidently report to the city government that illegal drug use causes homelessness: $X \rightarrow Y$, and therefore reducing drug use will reduce homelessness?

No, not without either deducing or collecting more information about the causal relationship between $X$ and $Y$. Perhaps it's the reverse, homelessness causes a higher risk for illegal drug use: $X \leftarrow Y$? Or perhaps $X$ and $Y$ are not as closely related, or even completely independent, and there's in fact a third variable such as mental illness ($Z$) that causes both homelessness and illegal drug use: $X \leftarrow Z \rightarrow Y$? In both of these cases the structure of your causal inference model will have to be altered from what you might see in a typical predictive model. There are many other possibilities as well (such as mediator and collider variables) which must be accounted for or ruled out in order to draw a complete picture of the cause and effect relationships.

Answer 5

Oo Oo! I'm a mathematical layperson! Let's see if I can do this:

---

TLDR: I use predictions (or "predictive models") to prepare for events beyond my control without having to know what actually causes them.

---

I might posit that a lay predictive model is "whether the weather report says it will rain this weekend". I may not care what will cause it to rain or not rain on this particular weekend, I cannot change it, and I only care about whether to pack my fishing equipment.

Contrast this with something I want to control: The weather in my house!!!

1. If I keep my roof in good shape, it probably will not rain.
2. If I set the thermostat for 74 degrees, it will probably stay between 72 and 76 degrees.
3. If I keep my windows closed, it will probably not be windy.

Etc.

I can depend on known, causal chains to control some things. But, I use predictive models to prepare for things beyond my control.

Answer 6

The basic problem is that a non-causal predictive model may fail if used for interventions. Start with a very simple example where an important part of the real world has:

$$ Cause \rightarrow \mathit{Effect} \rightarrow Measurement $$

Because we imagine Measurement is nearly perfect, the best predictive model will prefer it to a noisier Cause, but Measurement can only be known after Effect, so it cannot inform policies or interventions. Changing Measurement by some other means won't change Effect at all.

 ~ * ~ Possibly that's sufficient, but more detail below. ~ * ~

If Measurement is something like income or health or quality, you can of course measure it ahead of time, but then it's not really the right measurement, because it hasn't taken Effect into account yet. The predictions won't be as good. Here you would have done better to use Cause, if known.

Often Cause is not known, and Measurement could be as good a predictor as you like. In the ultimate example, we have entangled particles:

$$ \mathit{Spin\ 1} \leftarrow \mathit{Hidden\ Cause} \rightarrow \mathit{Spin\ 2} $$

In this case, Spin 2 is a perfect predictor of Spin 1, but it can't ever be used to change Spin 1: they both simply reflect a hidden common cause. In a less extreme example, ice-cream sales and drownings are both caused by heat, but also other things:

$$Other1 \rightarrow Sales \leftarrow \mathbf{Heat} \rightarrow Drownings \leftarrow Other2$$

We can't reduce drownings by banning ice-cream, and we can't do much about Heat, however predictive, but we can make policies about Other2. (And ice-cream sales could be an early warning allowing us to act.)

Usually there is no one perfect Measurement, so we manufacture one: our predictive model cleverly combines Many things. But even if it correctly accounts for dependencies among Many things, it doesn't help policy or interventions if Many things are downstream of Effect.

If Measurement is downstream of both Cause and Effect, then the model can fail in stranger ways, esp. if it was cleverly trying to control things statistically.

Answer 7

If it's a person who really knows barely anything about statistics and causation I would provide some examples that are straightforward.

If you have data on the number of bathrooms in someone's largest house, you can classify the person as a billionaire or not, in the sense that billionaire mansions have numerous bathrooms. I'd assume you could achieve some precise estimates in this problem. However, increasing the number of bathrooms in someone's house won't make them billionaires. So you could use such a model to identify billionaires, but not to make someone a billionaire. Correlation, but not causation.

Another common example is the ice cream consumption relationship to shark attacks (being confounded by temperature). In some places, you could get some reasonable prediction of shark attacks based on ice cream consumption. But throwing more sharks in beaches wouldn't make people eat more ice cream (at least not to the same extent you measured with the confounded estimate).

I agree with Graham Wright that this should not be a dichotomy. You can have predictions based on causal models, for example. If the person knows some statistics, I would go for a different approach, like the other answers, but if it's my not-even-high-school-diploma grandma, I would go for the intuitive examples I gave above.