2
$\begingroup$

We make a model of the following form:

$$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon $$

and with $n=1,000$, $\hat\beta_1$ has a p-value <0.001.

If our data and data collection meets assumptions of multiple linear regression, we can say:

  • If the population shows NO relationship between $x_1$ and $y$, then properly random samples of $n=1,000$ will show this degree of fit (or relationship) in <0.001 of the samples.

What then are the logically possible relationships between $x_1$ and $y$?

I see the following but want to know if there are others:

  • If $\beta_1 = 0$:
    1. $x_1$ and $y$ are linearly independent and uncorrelated
    2. $x_1$ and $y$ are two independent variables "rendered dependent" by observations on their effect, $x_2$
  • If $\beta_1 \ne 0$:
    1. $x_1$ is linear cause of $y$ (more umbrellas cause wetter sidewalks)
    2. $y$ is linear cause of $x_1$ (wetter sidewalks cause more umbrellas)
    3. $Z$ is linear cause of $x_1$ AND $y$ (more rain, GDP,... causes both)
    4. []
    5. $Cor(x_1,y) \ne 0$ and linearly dependent but no causal chain, parent, or child

Are the final three choices mutually exclusive and exhaustive if we are correct that $\beta_1 \ne 0$? Illustrative examples would help.

*Note, when I say "cause," I do not mean the direct, immediate cause, but instead that there is a shared causal chain from $l,m,n, ... \to x_1 \to p,q,r,... \to y$. Also, $\beta_1 \ne 0$. $\hat\beta_1$ is irrelevant.

Suggested additions that I dispute (but open to change):

  1. $x_1$ and $y$ are (jointly) linear cause(s) of $x_2$ (less heat and more water cause more time to boil water)
    • if this is not really a case of (3), hence new example, then it seems we incorrectly concluded that $\beta_1 \ne 0$ with two independent variables ($x_1$ and $y$) such as heat and water quantity. (I think "independent variables" $\to \beta_1 = 0 \to$ (10), not (4))
    • also violates multicollinearity assumption needed for $\hat{\beta_1} = \beta_1$ ($x_1$ and $x_2$ should be linearly independent and uncorrelated).
    • Berkson's paradox???
  2. $x_1$ and $y$ have no linear causal relationships at all (margarine consumption/capita $\to$ divorces in Maine/capita(k))
    • then it seems we incorrectly concluded that $\beta_1 \ne 0$. ($\beta_1 = 0 \to$ (10), not (5))
    • time-series autocorrelation (of values and/or errors) may pose a problem??
    • what is the population from which these values represent a random sample?
    • if this is the "population", $N=10$, how do we talk about it?
    • $Z \to x_1,y$ is true but unbelievable with current knowledge
$\endgroup$
  • 6
    $\begingroup$ No. Some readings that may help you understand how to weave causal inference into your statistical models: Greenland, S., Pearl, J., and Robins, J. M. (1999). Causal diagrams for epidemiologic research. Epidemiology, 10(1):37–48. and Maldonado, G. and Greenland, S. (2002). Estimating causal effects. International Journal of Epidemiology, 31(2):422–438. $\endgroup$ – Alexis Mar 11 '15 at 17:23
  • 4
    $\begingroup$ For more depth, see also: Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge University Press. And the forthcoming Hernán, M. A. and Robins, J. M. (2015). Causal Inference. Chapman & Hall/CRC. $\endgroup$ – Alexis Mar 11 '15 at 17:25
  • 3
    $\begingroup$ Note also that $\beta_1= 0$ is perfectly compatible with the idea that $x_1$ causes $x_2$, which causes $y$. $\endgroup$ – Scortchi - Reinstate Monica Mar 11 '15 at 18:10
  • 3
    $\begingroup$ @Scortchi: or just $x_1$ causes $y$, but not linearly. $\endgroup$ – Neil G Mar 11 '15 at 18:23
  • 1
    $\begingroup$ "If we correctly reject the null hypothesis...": Note that one would not be justified in saying that in the population B1 is 99.9% likely to be nonzero. One could say that if the null were true, 99.9% of sample B1's would be closer to zero than the observed one. $\endgroup$ – rolando2 Mar 12 '15 at 0:07
6
$\begingroup$

It sounds like you're asking if two variables $x_1, y$ are dependent, which causal relationships between them might account for the dependence.

You pointed out three. A fourth is the following where neither is the cause of the other, nor do they have a common cause:

$x_1 \rightarrow x_2 \leftarrow y$.

Or in general something like this. For example,

$x_1 \rightarrow a \leftarrow b \rightarrow c \leftarrow y$ where $a \rightarrow x_2 \leftarrow c$.

Your choices are not mutually exclusive because there could also be a common cause $z$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks! But I don't understand your arrows or your fourth possibility. Could you give an illustrative example of an $x_1$ and $y$ that are dependent and meet all assumptions as in original question that clearly is your (4) (or clearly does not fall into 1-3)? $\endgroup$ – jtd Mar 11 '15 at 20:19
  • 2
    $\begingroup$ @jtd: What don't you understand? If $x_1$ and $y$ are the causes of $x_2$, they will appear dependent given that $x_2$ is observed. $\endgroup$ – Neil G Mar 11 '15 at 20:33
  • $\begingroup$ (I should say, "may" appear dependent. They could nevertheless remain independent.) $\endgroup$ – Neil G Mar 11 '15 at 21:39
  • 1
    $\begingroup$ @jtd: Just rearrange the terms in your equation so that $x_2$ is a function of $y$ and $x_1$. Why is anything violated? Anyway, the linear relationship doesn't tell you anything about the actual direction of causality — just that things are dependent. $\endgroup$ – Neil G Mar 11 '15 at 23:12
  • 2
    $\begingroup$ @jtd but $\beta_1$ won't be zero because $x_1$ and $y$ won't be independent given observation of $x_2$. A pair of common causes can be rendered dependent by observation of their effect. $\endgroup$ – Neil G Mar 12 '15 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.