13
$\begingroup$

I am an MBA student taking courses in statistics.

I am sure that at some point, we have all heard the famous expression - "Correlation Does Not Imply Causality".

When we are being introduced to this concept, we are often shown an absurd example of two variables that are highly unlikely to have a causal relationship, yet seemingly appear to be correlated with one another. For example - it is highly unlikely that there might be a causal relationship between the "letters in the words of spelling bees" and the "number of people killed every year by spiders":

enter image description here

However, there are other examples in which it might be reasonable to assume that correlation does in fact imply causality. For example, it is quite likely that there might be some relationship between "life expectancy" and "GDP" - countries with higher GDP likely have stronger economies, stronger economies likely have stronger health sectors, countries with stronger health sectors likely have more hospitals/doctors ... using this logic, although there might be outliers and exceptions, it does not seem completely unreasonable to expect that there might be some causal correlation between income and life expectancy:

enter image description here

This leads me to my question - when we are told that "Correlation Does Not Imply Causality", does this statement indirectly mean that there might be some instances in which "Correlation DOES Imply Causality"?

For instance, suppose I have some data (for different countries) which shows a clear linear relationship between "the number of doctors per 1000 people" and the "life expectancy" of those countries (i.e. people who live in countries with more doctors tend to live longer on average) - although there some philosophical arguments that suggest we might never be able to "truly" attribute causality between two concepts (e.g. perhaps tomorrow we find out that countries with more doctors have some newly discovered vitamins in their water and this is why they live longer, not actually due to more available doctors, yet this is highly unlikely) - would be fundamentally wrong and morally unreasonable to prematurely suggest that "there theoretically might exist some level of causal relationship between the number of doctors and life expectancy"?

$\endgroup$
11
  • 10
    $\begingroup$ Judea Pearl has written extensively about causality and correlation, you might want to have a look at some of his books. $\endgroup$
    – F. Tusell
    Oct 8, 2022 at 6:31
  • 12
    $\begingroup$ Some of your confusion might come from the word "implies," which is used here in the formal mathematical sense of "if A then B" or "if A is true then B must also be true" (as opposed to the weaker way we use it in common speech to mean "suggests" or "hints at") $\endgroup$
    – A C
    Oct 8, 2022 at 17:36
  • 16
    $\begingroup$ Correlation correlates with causation. $\endgroup$
    – Tim
    Oct 8, 2022 at 18:08
  • 3
    $\begingroup$ An xkcd classic (don't miss the mouseover text): xkcd.com/552 $\endgroup$
    – mkt
    Oct 8, 2022 at 20:55
  • 4
    $\begingroup$ "yet seemingly appear to be correlated with one another" - well, they don't just appear to be correlated; they are correlated, measurably so. That's kind of the problem: spurious or misleading correlations are still correlations. $\endgroup$
    – kaya3
    Oct 9, 2022 at 2:06

5 Answers 5

24
$\begingroup$

I drew this slide a few years ago that might help enter image description here

Most of the silly correlations from that website are chance. Statistics is reasonably good at describing what can happen by chance, at least if you specify in advance the correlation you are interested in. The correlation between doctors and life expectancy is fairly robustly not explainable by chance.

The other possibilities on the slide all show correlation that's causal in nature, but only one of them is a simple 'blue causes red'. If you find doctors are correlated with life expectancy it could be

  • that doctors are actually good for health
  • increased life expectancy causes an increase in doctors (maybe because old people need them more?)
  • both the life expectancy and the increase in doctors are caused by something else. For example, maybe rich countries have more doctors (because doctors are expensive) and have better sanitation and nutrition (because sanitation and good nutrition are expensive) and that's the explanation
  • selection: you've somehow chosen a sample without the low-doctor/good-health and high-doctor/poor-health countries (it's easier to get good examples of this for negative correlation)

Deciding how much of a correlation is due to causation in the way that you're interested in and how much is due to causation in annoying and unhelpful ways is hard. The goal of modern causal inference is at least to render the question precise enough that you can say what you'd need to assume and measure to answer it.

[Update: One other possibility that's not in the slide is that two things are correlated because they're actually one thing, like the radius and area of circles, but that doesn't apply to your question]

$\endgroup$
8
  • $\begingroup$ Wrt slide: what about mediation? $\endgroup$
    – rep_ho
    Oct 8, 2022 at 7:48
  • $\begingroup$ if I may be a little leading, does identifying which scenario you are in require some additional information, outside the correlation ? $\endgroup$
    – Mike M
    Oct 8, 2022 at 15:21
  • $\begingroup$ @rep_ho: mediation is a different question, outside the scope of the slide. $\endgroup$ Oct 9, 2022 at 5:49
  • $\begingroup$ @MikeM Yes, absolutely. You have to know something about what else might affect number of doctors and life expectancy $\endgroup$ Oct 9, 2022 at 5:49
  • $\begingroup$ "they're actually one thing, like the radius and area of circles" could you please explain? $\endgroup$ Oct 9, 2022 at 8:16
20
$\begingroup$

The problem here is logical. Nobody can deny that causality can lead to correlation, and so there are instances in which you have causality and also correlation. The statement "correlation implies causality" however is a general logical statement; it means that you have causality whenever there is correlation, and the "whenever" doesn't make sense when discussing a single instance. So no, as long as counterexamples exist, correlation does not imply causality, and this statement is not contradicted by the existence of instances where you have both (or even where correlation can be legitimately taken to indicate the possibility of causality).

I add that one might think that even if there is no logical implication, there could be a strong probabilistic indication, i.e., when we see correlation, very often there is causation, and counterexamples are few and far between. This, however, can hardly be made precise (even before investigating this empirically, decisions would need to be made what to count, and it isn't at all obvious how to do that), and my impression is that there are for sure enough counterexamples and enough ways to explain how correlation arises without causation, that one should neither think that this "implication" holds "with high probability".

$\endgroup$
2
  • $\begingroup$ Yes, "logically implies" ($A \implies B$) is the result of the material implication ($A \rightarrow B$) being a tautology. $\endgroup$
    – Galen
    Oct 9, 2022 at 2:55
  • 1
    $\begingroup$ Indeed. I think the confusion is between the meaning of "imply" in logic (A can be logically deduced from B) and it's casual use in conversation to mean "makes one suspect that". $\endgroup$ Oct 11, 2022 at 8:13
2
$\begingroup$

To add to the excellent answers already given: the statement helps to create awareness of logical errors regarding correlation.

We all easily and intuitively accept there must be a relationship between causality and correlation. We feel that causality implies correlation. This is correct. Most people, including myself, have subsequent intuition that is logically flawed.

Personally, I have not succeeded in changing my intuition, which means that I constantly need to remind myself that I cannot reverse the argument, so I keep telling others but also mostly myself that correlation does not imply causality. Also, I need to remind myself that causality does not imply that I will be able to observe the correlation that I expected - for many reasons already mentioned in previous answers.

If it hasn’t settled in yet - the real problem is that our observations usually will only tell us if there is a correlation.

I therefore need to remind myself not to turn the argument on its head. I need to remind myself that observing a correlation when evaluating a hypothesis is much stronger than developing a hypothesis from an observed correlation. That does not mean I cannot use open observational studies to develop hypotheses (in fact, these studies can be very powerful for inspiration). it means that I need to test these developed hypotheses in carefully designed experiments, and that these experiments shall not use any of the previous observations that inspired the hypothesis. Most of the time, that implies a lot of work and therefore not a very appealing prospect.

I also need to remind myself to design my experiments such that I maximize the response to the supposed causal factor; or at least ensure that the response I expect is large enough to be observable. When that goes wrong, it is where it becomes easy for me but maybe not for others: I’m an engineer, so in my application, there is no practical difference between an unmeasurable response vs. no response at all. That means I can abandon the experiments at this point. Whereas more Fundamental scientific work may run into the need to measure even the small causal responses, and/or find some other way to figure out if the supposed causality is or isn’t there.

$\endgroup$
2
$\begingroup$

Spurious relationship

"Correlation Does Not Imply Causality" This saying relates in particular to the idea that correlation between two variables does not imply a causal relationship between the two variables. It does not always mean that there is an absence of causality.

A typical example of the 'Correlation Does Not Imply Causality' is the presence of a confounding variable and is explained by the correlation between 'ice cream sales' and 'shark attacks'. The two are correlated, but the one does not cause the other. Yet, there are causal relationships present. Both are caused by hot weather which makes more people buy ice cream and more people go to the beach.

This is your situation with the map of GDP per Capita and Life expectancy. There is most likely some sort of causal relationship present in that situation. The problem is however that the correlation alone does not allow you to determine what type of causal relationship is present. The phrase "Correlation Does Not Imply Causality" or the term spurious relationship relates to the situation where somebody falsely/falliciously concludes that there is a particular causal relationship based on a correlation.

Spurious correlation

Your case of the relationship between scripps and spiders is of a more specific nature and sometimes referred to as spurious correlation (see Misunderstandings of "spurious correlation"?). It means that one falsely assumes that there is a statistical correlation where there is none.

Examples may occur when data are correlated (e.g. time series Why do these time series appear to be dependent?) or when people make multiple comparisons without correcting.

$\endgroup$
3
  • $\begingroup$ @ Sextus: Thank you for these links! I will check them out! You seem to be very knowledgeable about Statistics - can you please check this out and tell me if I am "out to lunch" (i.e. crazy)? stats.stackexchange.com/questions/591643/… $\endgroup$
    – stats_noob
    Oct 9, 2022 at 16:55
  • 1
    $\begingroup$ @MBA_Grad_Student_2022 your idea in the question over there sounds reasonable to me, although I don't know that much about such machine learning problems to provide a nice answer. You can look into the methods how such regression models are actually computed, like bootstrap aggregating, and find out that many of the algorithms to fit these models actually don't require you to fully load the entire dataset all at once. $\endgroup$ Oct 9, 2022 at 18:13
  • $\begingroup$ Thank you! I will look into this more! $\endgroup$
    – stats_noob
    Oct 9, 2022 at 22:49
0
$\begingroup$

I think there are possibly two related issues causing confusion.

First, "imply" means to "suggest." And correlation is a clue that 2 things might be causally related. It doesn't prove it, but it hints that a cause might be there. So the statement "Correlation Does Not Imply Causality" is frankly wrong. Correlation does imply ("suggest") possible causation. The statement should be "Correlation Does Not Prove Causality."

Comments from Michael MacAskill and V C pointed out that "imply" in this case refers to logical implication: if p then q. In that context, the statement is definitely not "frankly wrong." However, the meaning is still "Correlation Does Not Prove Causality."

Which brings up the 2nd issue. When learning the difference between correlation and causation, students may get the impression that they are opposites or mutually exclusive. They are not. Correlation certainly does not rule out causation. If confounding variables are eliminated, then all causally related data will be correlated (not necessarily in a linear fashion), but not all correlated data will be causally related:

enter image description here

This is just another example that correlation does suggest (imply) but not prove causation. Or to put it another way, lack of correlation does prove lack of causation. Correlation is necessary but not sufficient to prove causation. (As long as confounding variables are controlled for. I'm being idealistic here for teaching purposes. It may not always be practical to control for confounds. Even if A and B are causally related, a scatterplot of A and B may still appear random if other factors also affect A and/or B.)

$\endgroup$
7
  • 11
    $\begingroup$ In formal notation, if A "implies" B then whenever A is true then B is guaranteed to be true. The expression is not wrong, it's just that people are more used to the vernacular usage of "imply" (a suggestion). The expression in question literally means correlation is not sufficient to prove causation. $\endgroup$
    – A C
    Oct 8, 2022 at 17:46
  • 1
    $\begingroup$ @A C. I see what you mean. Imply has two different means. Common vernacular means "suggest," but there is also the formal logical meaning of implication. But even then, logical implication says nothing about causality. $\endgroup$ Oct 8, 2022 at 17:52
  • 2
    $\begingroup$ This is a mathematical/statistical community. Giving an answer based on a lay understanding of the term implies is not helpful. In mathematics and statistics, implies most assuredly does not mean suggests. $\endgroup$ Oct 9, 2022 at 5:17
  • 1
    $\begingroup$ @Michael MacAskill. I agree with you and A C completely. That was a flaw with my analysis. I haven't edited the post because I did not come to that realization myself, and don't want to take credit for that correction. I'd like the comments to speak for themselves. I do think it's helpful for these 2 different meanings of "implies" to be pointed out, because I get the impression that the OP may be interpreting the term in the common sense and not the mathematical sense (as I did), and that's one of the things that may have lead to the question. Thank you for the feedback. $\endgroup$ Oct 9, 2022 at 16:24
  • 5
    $\begingroup$ Your "Venn diagram" is not quite correct. There are cases when A causes B, and yet A and B are uncorrelated. So that inner oval needs to protrude outside the outer oval. $\endgroup$ Oct 9, 2022 at 18:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.