Misunderstandings of "spurious correlation"?

Question

I've heard people use the term spurious correlation in so many different instances and various ways, that I'm getting confused. Moreover, the Wikipedia page for Spurious relationship states:

“In statistics, a spurious relationship or spurious correlation is a mathematical relationship in which two or more events or variables are not causally related to each other (i.e. they are independent), yet it may be wrongly inferred that they are, due to either coincidence or the presence of a certain third, unseen factor”

A couple of observations:

1. Mathematically speaking, two variables $A, B$ are independent $\iff P(A | B) = P(A)$, correct?

   Clearly, if two variables are correlated, even if the dependency is driven by some third factor, the two are still not independent, like the Wikipedia article claims. What's up with that?

2. If the “spurious” correlation is statistically significant (or not a result of coincidence), then what's wrong with that? I've seen people jumping out like rabid animals, foam coming out of their mouth screaming: “Spurious! Spurious!”.

   I don't understand why they do it — no one is claiming that there is a causal link between the variables. Correlation can exist without causation, so why label it “spurious”, which is sort of equivalent to calling it “fake”?

Answer 1

I've always hated the term "spurious correlation" because it is not the correlation that is spurious, but the inference of an underlying (false) causal relationship. So-called "spurious correlation" arises when there is evidence of correlation between variables, but the correlation does not reflect a causal effect from one variable to the other. If it were up to me, this would be called "spurious inference of cause", which is how I think of it. So you're right: people shouldn't foam at the mouth over the mere fact that statistical tests can detect correlation, especially if there is no assertion of an underlying cause. (Unfortunately, just as people often confuse correlation and cause, some people also confuse the assertion of correlation as an implicit assertion of cause, and then object to this as spurious!)

To understand explanations of this topic, and avoid interpretive errors, you also have to be careful with your interpretation, and bear in mind the difference between statistical independence and causal independence. In the Wikipedia quote in your question, they are (implicitly) referring to causal independence, not statistical independence (the latter is the one where $\mathbb{P}(A|B) = \mathbb{P}(A)$). The Wikipedia explanation could be tightened up by being more explicit about the difference, but it is worth interpreting it in a way that allows for the dual meanings of "independence".

Answer 2

First, correlation applies to variables but not to events, and so on that count the passage you quote is imprecise.

Second, "spurious correlation" has meaning only when variables are in fact correlated, i.e., statistically associated and therefore statistically not independent. So the passage is flawed on that count as well. Identifying a correlation as spurious becomes useful when, despite such a correlation, two variables are demonstrably not causally related to each other, based on other evidence or reasoning. Not only, as you say, can correlation exist without causation, but in some cases correlation may mislead one into assuming causation, and pointing out spuriosity is a way of combating such misunderstanding or shining a light on such incorrect assumptions.

Answer 3

There is a huge misunderstand about the meaning of "spurious correlation". Even among practitioners.

Spurious correlation is not only about absence of causal relation. It's about absence of correlation itself!

Spurious correlation appears when two totally uncorrelated variables present a correlation in-sample just by luck. Therefore, this is a concept closely related to the concept of type I error (when the null hypothesis assumes that X and Y are uncorrelated).

This distinction is very important because in some occasions what is relevant to know is if variables X and Y are correlated, no matter the causal relation. For example, for forecasting purpose, if the analyst observe X and X is correlated to Y, possibly X can be used to make a good forecast of Y.

A good paper that explore this concept is "Spurious regressions with stationary series" Granger, Hyung and Jeon. Link: https://escholarship.org/uc/item/7r3353t8 "A spurious regression occurs when a pair of independent series, but with strong temporal properties, are found apparently to be related according to standard inference in an OLS regression."

Summing up, we can have the following cases: (i) X causes Y or Y causes X; (ii) X and Y are correlated, but neither X causes Y nor Y causes X; (iii) X and Y are uncorrelated, but they present correlation in-sample by luck (spurious relation).

Answer 4

Let me try explaining the concept of spurious correlation in terms of graphical models. Generally, there is some hidden associated variable which is causing the spurious correlation.

Assume that the hidden variable is A and two variables which are spuriously correlated are B and C. In such scenarios, a graph structure similar to B<-A->C exist. B and C are conditionally independent (implies uncorrelated) which means B and C are correlated if A is not given and they are uncorrelated if A is given.

Answer 5

I don't think there's any real misunderstanding or confusion here. Yes, the Wikipedia entry could be more carefully worded and yes, "spurious correlation" is really about an improper interpretation of correlation. But ... So what?

Those of us who are data analysts/statisticians/data scientists or just geeky in this area understand this.

But the rest of the population will leap to conclusions from correlations all the time. Labeling some correlations as spurious is an attempt to limit those leaps. I think it's a better attempt than other phrases which are less dramatic.

In a certain sense, the correlation is spurious or fake and "foaming at the mouth" is sometimes the appropriate reaction. All sorts of bad decisions have been made by people who don't understand (or simply forget) that correlation is not causation.