# Does rejection of null hypothesis in multiple regression entail causation?

We make a model of the following form:

$$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon$$

and with $n=1,000$, $\hat\beta_1$ has a p-value <0.001.

If our data and data collection meets assumptions of multiple linear regression, we can say:

• If the population shows NO relationship between $x_1$ and $y$, then properly random samples of $n=1,000$ will show this degree of fit (or relationship) in <0.001 of the samples.

What then are the logically possible relationships between $x_1$ and $y$?

I see the following but want to know if there are others:

• If $\beta_1 = 0$:
1. $x_1$ and $y$ are linearly independent and uncorrelated
2. $x_1$ and $y$ are two independent variables "rendered dependent" by observations on their effect, $x_2$
• If $\beta_1 \ne 0$:
1. $x_1$ is linear cause of $y$ (more umbrellas cause wetter sidewalks)
2. $y$ is linear cause of $x_1$ (wetter sidewalks cause more umbrellas)
3. $Z$ is linear cause of $x_1$ AND $y$ (more rain, GDP,... causes both)
4. []
5. $Cor(x_1,y) \ne 0$ and linearly dependent but no causal chain, parent, or child

Are the final three choices mutually exclusive and exhaustive if we are correct that $\beta_1 \ne 0$? Illustrative examples would help.

*Note, when I say "cause," I do not mean the direct, immediate cause, but instead that there is a shared causal chain from $l,m,n, ... \to x_1 \to p,q,r,... \to y$. Also, $\beta_1 \ne 0$. $\hat\beta_1$ is irrelevant.

Suggested additions that I dispute (but open to change):

1. $x_1$ and $y$ are (jointly) linear cause(s) of $x_2$ (less heat and more water cause more time to boil water)
• if this is not really a case of (3), hence new example, then it seems we incorrectly concluded that $\beta_1 \ne 0$ with two independent variables ($x_1$ and $y$) such as heat and water quantity. (I think "independent variables" $\to \beta_1 = 0 \to$ (10), not (4))
• also violates multicollinearity assumption needed for $\hat{\beta_1} = \beta_1$ ($x_1$ and $x_2$ should be linearly independent and uncorrelated).
2. $x_1$ and $y$ have no linear causal relationships at all (margarine consumption/capita $\to$ divorces in Maine/capita(k))
• then it seems we incorrectly concluded that $\beta_1 \ne 0$. ($\beta_1 = 0 \to$ (10), not (5))
• time-series autocorrelation (of values and/or errors) may pose a problem??
• what is the population from which these values represent a random sample?
• if this is the "population", $N=10$, how do we talk about it?
• $Z \to x_1,y$ is true but unbelievable with current knowledge
• No. Some readings that may help you understand how to weave causal inference into your statistical models: Greenland, S., Pearl, J., and Robins, J. M. (1999). Causal diagrams for epidemiologic research. Epidemiology, 10(1):37–48. and Maldonado, G. and Greenland, S. (2002). Estimating causal effects. International Journal of Epidemiology, 31(2):422–438. Mar 11 '15 at 17:23
• For more depth, see also: Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge University Press. And the forthcoming Hernán, M. A. and Robins, J. M. (2015). Causal Inference. Chapman & Hall/CRC. Mar 11 '15 at 17:25
• Note also that $\beta_1= 0$ is perfectly compatible with the idea that $x_1$ causes $x_2$, which causes $y$. Mar 11 '15 at 18:10
• @Scortchi: or just $x_1$ causes $y$, but not linearly. Mar 11 '15 at 18:23
• "If we correctly reject the null hypothesis...": Note that one would not be justified in saying that in the population B1 is 99.9% likely to be nonzero. One could say that if the null were true, 99.9% of sample B1's would be closer to zero than the observed one. Mar 12 '15 at 0:07

It sounds like you're asking if two variables $x_1, y$ are dependent, which causal relationships between them might account for the dependence.

You pointed out three. A fourth is the following where neither is the cause of the other, nor do they have a common cause:

$x_1 \rightarrow x_2 \leftarrow y$.

Or in general something like this. For example,

$x_1 \rightarrow a \leftarrow b \rightarrow c \leftarrow y$ where $a \rightarrow x_2 \leftarrow c$.

Your choices are not mutually exclusive because there could also be a common cause $z$.

• Thanks! But I don't understand your arrows or your fourth possibility. Could you give an illustrative example of an $x_1$ and $y$ that are dependent and meet all assumptions as in original question that clearly is your (4) (or clearly does not fall into 1-3)?
– jtd
Mar 11 '15 at 20:19
• @jtd: What don't you understand? If $x_1$ and $y$ are the causes of $x_2$, they will appear dependent given that $x_2$ is observed. Mar 11 '15 at 20:33
• (I should say, "may" appear dependent. They could nevertheless remain independent.) Mar 11 '15 at 21:39
• @jtd: Just rearrange the terms in your equation so that $x_2$ is a function of $y$ and $x_1$. Why is anything violated? Anyway, the linear relationship doesn't tell you anything about the actual direction of causality — just that things are dependent. Mar 11 '15 at 23:12
• @jtd but $\beta_1$ won't be zero because $x_1$ and $y$ won't be independent given observation of $x_2$. A pair of common causes can be rendered dependent by observation of their effect. Mar 12 '15 at 16:01