After we have shown that two quantities are correlated how do we infer that the relationship is causal? And furthermore which one causes what? Now in theory one can use a "random assignment" (whatever the right word is), to break any accident bonds that may exist between two variables. But in some cases this is not possible to do. For example, consider how often a person smokes, measured in cigarettes per week, vs the life expectancy, measured in years. We can randomly pick two groups of people. Make one group smoke and the other not do. As the assignment is random this should break any other relations between them. But this is of course not possible to do for many different reasons. So what are some of the techniques that can be used?

  • 1
    $\begingroup$ Through carefully planned experiments. ;-) $\endgroup$ – StatsStudent Feb 18 '15 at 18:59
  • $\begingroup$ @StatsStudent What experiments? Take for example, cigarettes vs life expectancy. Do you really want to do that experiment, under some kind of control condition, if you think they lower life expectancy? With experiments it is easy to verify causation. But how does one do it from a correlation plot? $\endgroup$ – Nicolas Bourbaki Feb 19 '15 at 9:08
  • $\begingroup$ @NicolasBourbaki your question starts by assuming the quantities are correlated. Does this imply that we are also assuming that the variables are related in a linear manner, such as Y=A*X+B? $\endgroup$ – cantorhead Mar 5 '16 at 23:08
  • $\begingroup$ @NicolasBourbaki One could define $Y(t+1)=\cos(X(t))-1+ noise$ and many would think of $X$ as "causing" $Y$. On the other hand $X(t)$ and $Y(t+1)$ are not correlated. $\endgroup$ – cantorhead Mar 5 '16 at 23:15
  • $\begingroup$ @NicolasBourbaki I have provided an answer assuming linearity below and would like to provide a more general answer but it would be off topic if you are only interested in linear relationships. $\endgroup$ – cantorhead Mar 5 '16 at 23:24

I think this is a very good question. I encounter this problem often and reflect on it a lot. I do research in medical science and the notion in medicine is that nothing is proven causal, never, never, never, until an randomized clinical controlled trial, preferably with a pill (or any other exposure that can be triple-blinded), have proven an effect on the response of interest. This is quite sad, as all other studies are considered to be association studies, which tend to reduce their impact.

Hill and Richard Doll thought about this. The former formulated Hill's criteria for causality:

The Bradford Hill criteria, otherwise known as Hill's criteria for causation, are a group of minimal conditions necessary to provide adequate evidence of a causal relationship between an incidence and a consequence, established by the English epidemiologist Sir Austin Bradford Hill (1897–1991) in 1965.

Strength: A small association does not mean that there is not a causal effect, though the larger the association, the more likely that it is causal. Consistency: Consistent findings observed by different persons in different places with different samples strengthens the likelihood of an effect. Specificity: Causation is likely if a very specific population at a specific site and disease with no other likely explanation. The more specific an association between a factor and an effect is, the bigger the probability of a causal relationship. Temporality: The effect has to occur after the cause (and if there is an expected delay between the cause and expected effect, then the effect must occur after that delay). Biological gradient: Greater exposure should generally lead to greater incidence of the effect. However, in some cases, the mere presence of the factor can trigger the effect. In other cases, an inverse proportion is observed: greater exposure leads to lower incidence. Plausibility: A plausible mechanism between cause and effect is helpful (but Hill noted that knowledge of the mechanism is limited by current knowledge). Coherence: Coherence between epidemiological and laboratory findings increases the likelihood of an effect. However, Hill noted that "... lack of such [laboratory] evidence cannot nullify the epidemiological effect on associations". Experiment: "Occasionally it is possible to appeal to experimental evidence". Analogy: The effect of similar factors may be considered.

This was formulated some 50 years ago, before the advent of randomized trials (which might not be of interest to your particular field) but it is noteworthy that experiments were not given a crucial role in the Hill criteria.

I'd like to think that observational data, if analysed with proper statistical methods, does allow for inferences of causality. (Of course this depends on many factors.) But in my field, when it comes to changing management of patients, it is rare to see guidelines shaped by anything other than randomized trials and the prelude to guidelines often underline that certain causality can only be obtained in randomized trials.

Now I know that many of you will not agree with me. I don't agree with myself neither. But it might add to a discussion.

  • $\begingroup$ "(which might not be of interest to your particular field)" My interest is algebraic geometry and arithmetic. Which is as far as removed from statistics as one can possibly imagine. I only ask it as a curiosity. $\endgroup$ – Nicolas Bourbaki Feb 15 '15 at 20:48

Statistics provides tools for detecting and modelling regularities in the data. The modelling process is typically guided by subject-matter knowledge. When the model represents the subject-matter mechanism, statistical properties of the estimated model tell whether the data is at odds with the modelled mechanism. Then causality (or lack thereof) is inferred -- and this is done on the subject-matter domain.

An example: suppose you have a random sample of two variables $x$ and $y$. The correlation between them is large and statistically significant.

So far, can you say whether $x$ causes $y$? I don't think so.

Now add subject-matter knowledge to the data.
Case A: the observed variables are length of feet and favourite shoe size
$\rightarrow$ people like buying shoes that fit their feet size, so feet size causes the choice of shoe size (but not the other way around).
Case B: the observed variables are height and weight of people
$\rightarrow$ adults tend to be both taller and heavier than kids, but does that mean weight causes height or height causes weight? Genetics, nutrition, age and other factors cause both.


The question currently assumes that the quantities are correlated, which implies that the person determining the correlation must have good reason to believe the variables share a linear relationship.

Granger Causality might be the best tool for determining linear causal relationships. Granger was an economist who shared a nobel prize for his work on linear Causation.

Granger suggests that for a set of variables $\{X_t^{(i)}\}_{i=1}^k$ to be considered a cause for effect $Y_t$,two conditions should hold:

  1. The cause should occur before the effect.
  2. The cause should contain information about the effect that is not available otherwise.

To find the shared information one can use regression (although beware that significant regression coefficients do not imply shared information in theory -- just in practice). Specifically, one wants to compare the residuals with and without the cause variables. Consider the variables to be column vectors, so that $\mathcal{X}=[X_{t-1}^{(1)},X_{t-2}^{(1)},\ldots,X_{t-m}^{(1)},X_{t-1}^{(2)},X_{t-2}^{(2)},\ldots,X_{t-m}^{(2)},\ldots,X_{t-m}^{(k)}]^T$ is also a column vector, and $\mathcal{Y}=[Y_{t-1},Y_{t-2},\ldots,Y_{t-m}]^T$ is a column vector. ($m$ is called the order or the time lag. There are methods to optimally chose $m$, but I think people just guess the best $m$ or base it on other constraints.) Then the regression equations of interest are \begin{align*} Y_t=A\cdot\mathcal{Y}+\epsilon_t \\ Y_t=A'\cdot[\mathcal{Y},\mathcal{X}]^T+\epsilon'_t. \end{align*} To determine if the $X_{t-i}^{(j)}$ contained info about $Y_t$ one would do an F-test on the variances of $\epsilon_t$ and $\epsilon'_t$.

To ensure that the information is not accounted for by any other source, one would gather up every other variable that can be accounted for, say $Z_t^{(1)},\ldots,Z_t^{(p)}$, define $\mathcal{Z}=[Z_{t-1}^{(1)},Z_{t-2}^{(1)},\ldots,Z_{t-m}^{(p)}]^T$, and do the regression \begin{align*} Y_t=A\cdot[\mathcal{Y},\mathcal{Z}]^T+\epsilon_t \\ Y_t=A'\cdot[\mathcal{Y},\mathcal{X},\mathcal{Z}]^T+\epsilon'_t. \end{align*} and do the same F-test on the residuals.

This is just a rough sketch and I believe that many authors have improved upon this idea.

  • $\begingroup$ Welcome to the site, @cantorhead. We want (questions and) answers here to be self-contained. It would be better if you tried "to be more specific here" & didn't just suggest people Google GC. $\endgroup$ – gung - Reinstate Monica Mar 5 '16 at 23:46

You can't--at least not within statistics.

Maxim: you can never know for certain that the effect of one variable is caused by another. The reason: you can never know if there's not another variable that you are not aware of and the data you've collected can't possibly tell you.

The fact of life is that data collection isn't always sufficient when data is static and the phenomenon is dynamic--like human behavior. There the collection of data itself can skew results, just like how in particle physics the fact of observation itself can't be removed from the equation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.