27
$\begingroup$

A pet saying of many statisticians is "Correlation doesn't imply causation." This is certainly true, but one thing that DOES seem implied here is that correlation has little or no value. Is this true? Is it useless to have knowledge that two variables are correlated?

I can't imagine that is the case. I'm not horribly familiar with predictive analysis, but it seems that if X is a predictor of Y, it would be useful in predicting future values of Y based on X, regardless of causality.

Am I incorrect in seeing value in correlation? And if not, in what situations might a statistician or data scientist use correlation without causation?

$\endgroup$
  • 5
    $\begingroup$ In my opinion, the phrase "causation does not imply correlation" is often misused to imply that statistics should not always be trusted (true, but not always due to lack of causality). I get so aggravated when I see people drop this phrase in reference to why a predictive analysis is wrong. For example, facebook.com/notes/mike-develin/debunking-princeton/… is great example of both a terrible analysis AND a terrible debunking of a terrible analysis. $\endgroup$ – Cliff AB Jul 24 '15 at 21:37
  • 10
    $\begingroup$ For instance, suppose you find that living in a certain city is correlated with early death. You can't conclude that living in that city causes early death, nor that getting people to move out of that city would help them live longer. (Maybe the city is attractive to sickly people, for some reason.) But if you are an actuary, you would be perfectly justified in wanting to charge higher life insurance premiums to members of that city - knowing about this correlation could be very valuable to you. $\endgroup$ – Nate Eldredge Jul 25 '15 at 0:24
  • 2
    $\begingroup$ More people die in the south of England, @NateEldredge. That's because people retire there. $\endgroup$ – TRiG Jul 26 '15 at 20:55
  • 1
    $\begingroup$ The absence of correlation carries more meaning, arguably. $\endgroup$ – Raphael Jul 27 '15 at 10:49
  • $\begingroup$ Mandatory xkcd reference: xkcd.com/552 $\endgroup$ – vsz Jul 28 '15 at 6:24
31
$\begingroup$

Correlation (or any other measure of association) is useful for prediction regardless of causation. Suppose that you measure a clear, stable association between two variables. What this means is that knowing the level of one variable also provides you with some information about another variable of interest, which you can use to help predict one variable as a function of the other and, most importantly, take some action based on that prediction. Taking action involves changing one or more variables, such as when making an automated recommendation or employing some medical intervention. Of course, you could make better predictions and act more effectively if you had more insight into the direct or indirect relationships between two variables. This insight may involve other variables, including spatial and temporal ones.

$\endgroup$
  • 4
    $\begingroup$ Correlations are not always useful for prediction. In cases of reverse causation, there are important temporal aspects that can't always be controlled for. We are running into this all the time with Alzheimer's Disease. We are constantly hitting our head against the wall trying to discern: are the biomarkers we find in AD affected brains causing the disease or caused by the disease? $\endgroup$ – AdamO Jul 25 '15 at 15:26
  • 1
    $\begingroup$ @AdamO I think my answer covers that base in the last sentence or two, so I do not disagree with you. $\endgroup$ – Brash Equilibrium Jul 25 '15 at 20:56
  • 1
    $\begingroup$ The problem with causality actually arises only if you are trying to interpret your predictive model. (Of course this is what we are often interested in science). When we see that biomarker A is a very good predictor it is very tempting to claim that this is also the cause of the disease - And as mentioned in the comments, it is very easy to come to wrong conclusions. If we only want to make predictions, e.g. tell whether a patient has the disease or not, there are no problems with correlations. $\endgroup$ – cel Jul 27 '15 at 11:26
  • $\begingroup$ This is untrue and here is but one example why. If acting on your predictions involves changing a variable and expecting the target to also change, but there is in fact no direct link or the causal relationship goes the other way, then you will take the wrong action. And before you say, "but in that example you are interpreting the model," I say, "in what scenario would you NOT draw inference even from a model meant for prediction?" Answer: when you don't put much trust in the causal relationships that your model implies. $\endgroup$ – Brash Equilibrium Jul 27 '15 at 14:11
  • $\begingroup$ @BrashEquilibrium: There are plenty of ways to act on a prediction that don't involve altering the variables used to obtain the prediction in any way. Interested in knowing whether your store should stock up on wool mittens? Knowing how much ice cream you've been selling lately could (in the hypothetical absence of more direct sources of data, of course) make a good predictor. $\endgroup$ – Ilmari Karonen Jul 27 '15 at 15:18
17
$\begingroup$

There are a lot of good points here already. Let me unpack your claim that "it seems that if X is a predictor of Y, it would be useful in predicting future values of Y based on X, regardless of causality" a little bit. You are correct: If all you want is to be able to predict an unknown Y value from a known X value and an known, stable relationship, the causal status of that relationship is irrelevant. Consider that:

  • You can predict an effect from a cause. This is intuitive and uncontroversial.
  • You can also predict a cause from knowledge of an effect. Some, but very few, people who get lung cancer never smoked. As a result, if you know someone has lung cancer, you can predict with good confidence that they are / were a smoker, despite the fact that smoking is causal and cancer is the effect. If the grass in the yard is wet, and the sprinkler hasn't been running, you can predict that it has rained, even though rain is the cause and wet grass is just the effect. Etc.
  • You can also predict an unknown effect from a known effect of the same cause. For example, if Billy and Bobby are identical twins, and I've never met Billy, but I know that Bobby is 5' 10' (178 cm), I can predict Billy is also 178 cm with good confidence, despite the fact that neither Billy's height causes Bobby's height nor does Bobby's height cause Billy's height.
$\endgroup$
  • 7
    $\begingroup$ Just to give names to your categories: Your three kinds of prediction are called (in order) deduction, abduction, and induction. $\endgroup$ – Neil G Jul 25 '15 at 0:11
12
$\begingroup$

They aren't poopooing the importance of correlation. It's just that the tendency is to interpret correlation as causation.

Take breastfeeding as the perfect example. Mothers almost always interpret the (observational studies') findings about breastfeeding as a suggestion as to whether or not they should actually breastfeed. It's true that, on average, babies who are breastfed tend to be healthier adults in order age even after controlling for longitudinal maternal and paternal age, socioeconomic status, etc. This does not imply that breastfeeding alone is responsible for the difference, though it may partially play a role in early development of appetite regulation. The relationship is very complex and one can easily speculate at a whole host of mediating factors that could underlie the differences observed.

Plenty of studies look to associations to warrant a deeper understanding of what's going on. Correlation is not useless, it just is several steps below causation and one needs to be mindful of how to report findings to prevent misinterpretation from nonexperts.

$\endgroup$
9
$\begingroup$

You're right that correlation is useful. The reason that causal models are better than associational models is that — as Pearl says — they are oracles for interventions. In other words, they allow you to reason hypothetically. A causal model answers the question "if I were to make X happen, what would happen to Y?"

But you do not always need to reason hypothetically. If your model is only going to be used to answer questions like "if I observe X, what do I know about Y?", then an associational model is all you need.

$\endgroup$
  • 3
    $\begingroup$ Oracles For Interventions would be a good name for a band. $\endgroup$ – Malvolio Jul 25 '15 at 20:51
  • $\begingroup$ @Malvolio: lol, it is an unforgettably succinct way to describe causal models. I really like that phrase. $\endgroup$ – Neil G Jul 25 '15 at 22:23
4
$\begingroup$

You are correct that correlation is useful for prediction. It is also useful for getting a better understanding of the system under study.

One case where knowledge about the causal mechanism is necessary is if the target distribution has been manipulated (e.g. some variables have been "forced" to take certain values). A model based on correlations only will perform poorly, whereas a model that used causal information should perform much better.

$\endgroup$
2
$\begingroup$

Correlation is an useful tool if you have an underlying model that explains causality.

For example if you know that applying a force to an object influences its movement, you can measure the correlation between the force and velocity and force and acceleration. The stronger correlation (with the acceleration) will be explanatory by itself.

In observational studies, correlation can reveal certain common patterns (as stated breastfeeding and later health) which might be give a ground for further scientific exploration via proper experimental design that can confirm or reject causality (e.g. maybe instead of breastfeeding being the cause it might be the consequence for a certain cultural framework).

So, correlation can be useful, but it can rarely be conclusive.

$\endgroup$
2
$\begingroup$

As you stated, correlation alone has plenty of utility, mainly prediction.

The reason this phrase is used (or misused, see my comment up top to the post) so often is that causation is often a much more interesting question. That is to say, if we've spent a lot of effort to examine the relation between $A$ and $B$, it is very likely because, back in the real world, we are curious if we can use $A$ to to influence $B$.

For example, all these studies showing that heavy usage of coffee in senior citizens is correlated with healthier cardio-vascular systems are, in my mind, undoubtable motivated by people wanting to justify their heavy coffee habits. However, saying drinking coffee is only correlated with healthier hearts, rather than causal, does nothing to answer our real question of interest: are we going to be healthier if we drink more coffee or if we cut down? It can be very frustrating to find very interesting results (Coffee's linked to healthier hearts!) but not be able to use that information to make decisions (still don't know if you should drink coffee to be healthier), and so there's almost always a temptation to interpret correlation as causation.

Unless maybe all you care about is gambling (i.e. you want to predict but not influence).

$\endgroup$
2
$\begingroup$

There's value in correlation, but one should look at more evidence to conclude causation.

Years ago, there was a study resulting in "coffee causes cancer." As soon as I heard this on the news I told my wife "false correlation." It turned out I was correct. The 2-3 cup per day coffee population had a higher rate of smoking than the non-coffee drinkers. Once the data collectors figured this out, they retracted their results.

Another interesting study before the housing boom and bust showed racism when it came to processing mortgages. The claim was that black applicants were being rejected at a higher rate than whites. But another study looked at default rates. Black homeowners were defaulting at the sames rate as whites. If black application were being held to a higher standard, their default rate would actually be far lower. Note: this anecdote was shared by author Thomas Sowell in his book The Housing Boom and Bust

Data mining can easily produce two sets of data that show high correlation, but for events that couldn't possibly be related. In the end, it's best to look at studies that are sent your way with a very critical eye. Finding false correlations isn't always easy, it's an acquired talent.

$\endgroup$
  • $\begingroup$ I enjoyed reading this answer. It seems, though, to address the inverse of the question: "Is it useless to have knowledge that two variables are correlated? ... In what situations might a statistician or data scientist use correlation without causation?" $\endgroup$ – whuber Jul 27 '15 at 17:18
  • 1
    $\begingroup$ "Black homeowners were defaulting at the sames rate as whites. If black application were being held to a higher standard, their default rate would actually be far lower." is jumping to conclusions. It's exactly this problem; black applicants are statistically going to be different from white applicants, and if more blacks are in a group that are more likely to have accepted mortgages default, black applicants having the same default rate would indicate discrimination against. Separating out confounding effects is hard. $\endgroup$ – prosfilaes Jul 28 '15 at 5:25
  • $\begingroup$ As I stated, the anecdote came from a well known black scholar. And it took far more that a paragraph to discuss in the book I referenced. $\endgroup$ – JoeTaxpayer Jul 28 '15 at 11:06
1
$\begingroup$

Correlation is an observable phenomenon. You can measure it. You can act on those measurements. On its own, it can be useful.

However, if all you have is a correlation, you do not have any guarantee that a change you make will actually have an effect (see the famous graphs tying the rise of iPhones to overseas slavery and such). It just shows that there is a correlation there, and if you tweak the environment (by acting), that correlation may still be there.

However, this is a very subtle approach. In many scenarios we want to have a less subtle tool: causality. Causality is a correlation combined with a claim that if you tweak your environment by acting in one way or another, one should expect the correlation to still be there. This allows for longer term planning, such as the chaining of 20 or 50 causal events in a row to identify a useful outcome. Doing so with 20 or 50 correlations often leaves a very fuzzy and murky result.

As an example of how they have been useful in the past, consider western science vs. Traditional Chinese Medicine (TCM). Western science focuses primarily on "Develop a theory, isolate a test which can demonstrate the theory, run the test and document the results." This starts with "develop a theory," which is highly tied to causality. TCM spun it around, starting with "devise a test which may provide useful results, run the test, identify correlations in the answer." The focus is more on correlations.

Nowdays westerners tend to prefer to think almost entirely in causality terms, so the value of studying correlation is harder to spy. However, we find it lurking in every corner of our life. And never forget that even in western science, correlations are an important tool for identifying which theories are worth exploring!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.