4
$\begingroup$

What are the relationships between correlation, dependence and causality?

I know that non-zero correlation does not imply causality. Though obviously causality implies correlation.

I also know that independence implies zero correlation.
By logical contraposition, if $P \rightarrow Q$, then $\neg Q \rightarrow \neg P$.
Does it mean then that non-zero correlation implies dependence?

What is the relationship between dependence and causality?
Does causality imply dependence?

Improve this question
$\endgroup$
9
  • 2
    $\begingroup$ "Obviously causality implies correlation:" that isn't so obvious. In fact, feedback loops are designed to prevent just that! A technical nicety--but an important one--is that correlation doesn't necessarily exist, so in that sense independence doesn't always imply zero correlation, either. But that doesn't ruin your argument: nonzero (but finite, defined) correlation and independence cannot both occur. $\endgroup$
    – whuber ♦
    Feb 11, 2021 at 22:55
  • $\begingroup$ Causality implies mutual information. Might be a correlation, might not. $\endgroup$
    – Adrian Keister
    Feb 12, 2021 at 3:43
  • 1
    $\begingroup$ @AdrianKeister, what if there are compensating effects such that two causes of equal strength and opposite effect yield the outcome unchanged? $\endgroup$
    – Richard Hardy
    Feb 12, 2021 at 6:50
  • 1
    $\begingroup$ Regarding though obviously causality implies correlation (which is incorrect), let me quote Mark Twain: It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so. One of my professors used to say: if you want to find a mistake in a proof, look at the place where it says trivially or obviously or clearly :) $\endgroup$
    – Richard Hardy
    Feb 12, 2021 at 6:53
  • $\begingroup$ Yes, I understand now that causality does not imply correlation. For example if we have pseudo-random generator, which takes a number and spits out another evenly distributed number, then we have no correlation, but we know that pseudo-random generator "caused" the random number. Another less extreme example would be a sinusoidally related physical variables, where independent value is directly related to the dependent value, but the correlation of the points on the sine curve is zero. My main question though is "does non-zero correlation imply dependence?". $\endgroup$
    – Reemet
    Feb 12, 2021 at 7:19

1 Answer 1

Reset to default
3
$\begingroup$

Let's first tackle dependence versus correlation and then we'll add causation into the mix. There are exceptions to almost every rule, some of them worth only a minimal footnote and you should start worrying about those exceptions only after you've been exposed to what is true as a rule.

Correlation is one type of dependence. Dependence is a broader concept. Correlation is a measure of linear dependence only. Two random variables are independent if they are not correlated and not dependent in any other way. Intuitively, two random variables are independent if knowledge that one has taken a particular value tells us absolutely nothing we didn't already know about the probability distribution of the other variable. Therefore, two random variables that are correlated are, by definition, dependent. With correlated variables, knowledge that one random variable has taken a particular value does tell us something new about the probability distribution of the other/correlated variable.

In practice, two causally related variables will be dependent as a rule, and a large fraction of those times they will be correlated. Causality is manifested as structural dependence. If you have drawn the correct DAG for the model and you are conditioning on the correct set of variables, if your data is without material errors, if you happen to have the correct functional form for the dependence then you will find correlation between the cause and effect variables. That is a lot of if's. Nevertheless, in practice two causally related variables will tend to be dependent, and a large fraction of those times they will be correlated.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.