regression and causation

Question

In the Chen and Pearl (2013) article there are several critics about econometrics textbooks. Currently I try to understand more about it. In particular the Authors written (pag 4, footnote 5):

From a causal analytic perspective, $X$ is exogeneous if $E[Y |X] = E[Y |do(X)]$ (Pearl, 2000). However, for purposes of this paper, we will use the aforementioned defnition in which $X$ is exogenous if it is independent of $e$. Note that if $X$ is independent of $e$ then $E[Y |X] = E[Y |do(X)]$. The converse may not hold. For example, when $e$ is a vector of factors with cancelling influences on $Y$ .

If I understand correctly "e", at least in general, is a structural error and not regression residual.

The structural model is $y=bX+ e$.

Then if $X$ is independent of $e$, the regression residual of a regression (OLS world) write with the same form of structural model are equal to structural error and the slope parameter have causal meaning. (conditional expectation is always true for OLS regression)

My question is: all structural model apart, if the residuals regression are completely independent of regressors, not only uncorrelated (true by costruction), parameters have a causal meaning? Or residual say nothing in itself about causality even under independance?

Answer 1

My question is: all structural model apart, if the residuals regression are completely independent of regressors, not only uncorrelated (true by costruction), parameters have a causal meaning?

No, let me show you a simple counter-example. Consider the SCM:

$$ Y = U_{y}\\ X = U_{x} $$

Where the density of the structural error terms $f(U_{y},U_{x})$ is a bivariate gaussian with mean zero, unit variance and covariance $\sigma_{u_{x}u_{y}}$. Here, trivially, neither $X$ nor $Y$ cause each other, that is $E[Y|do(x)] =E[Y]$ and $E[X|do(y)]=E[X]$.

Yet, the regression coefficient of $X$ on $Y$ is $\sigma_{u_{x}u_y}$. Now since the regression residual is constructed as $\epsilon = Y-\sigma_{u_{x}u_y}X$, it is normally distributed (linear combination of normals). Also by construction, $\epsilon$ is uncorrelated with $X$, since $cov(\epsilon, X) = cov(Y, X) - \sigma_{u_{x}u_y} = 0$. But this also implies they are independent, since $\epsilon$ and $X$ are joint normal (no correlation implies independence in multivariate normal distributions).

So here I showed you an example where the residual is completely independent of the regressor, yet the regression coefficient has no causal meaning. But more generally, you should internalize the following mantra: "no causes in, no causes out". It's impossible to make causal inference without causal assumptions, so whenever you wonder whether it's possible for a statistical quantity to have causal meaning, but your set of assumptions make reference only to the joint probability distribution, you can be assured the answer is "no".

Answer 2

There is a mistake in the answer that gives an example to try to show you can have $X$ and $\varepsilon$ independent, but the relationship between $X$ and $Y$ is not causal: $\sigma_{\mu_x\mu_y}$ is only the regression coefficient if $cov(X,\varepsilon)=0$ so it cannot be used to show $cov(X,\varepsilon)=0$

Since $X$ and $Y$ are joint normal the only dependence $Y$ can have on $X$ is a linear one, so we state that as

$Y=\beta_0+\beta_1X+\varepsilon$ where $E(\varepsilon)=0$

We can find an expression for $\beta_1$ as follows

$cov(X,Y)=cov(X,\beta_0+\beta_1X+\varepsilon)\\=cov(X,\beta_0)+cov(X,\beta_1X)+cov(X,\varepsilon)=0+\beta_1cov(X,X)+cov(X,\varepsilon)\\=\beta_1var(X)+cov(X,\varepsilon)$

solving for $\beta_1$ we get

$\beta_1=\frac{cov(X,Y)}{var(X)}-\frac{cov(X,\varepsilon)}{var(X)}$

from which we see that $\beta_1=\frac{cov(X,Y)}{var(X)}$ only if $cov(X,\varepsilon)=0$

and if $var(X)=1$ (as was assumed in the first answer) then we have

$\beta_1=cov(X,Y)$ only if $cov(X,\varepsilon)=0$

I changed notation: (relating back to first answer: $cov(X,Y)=\sigma_{\mu_x\mu_y}$ and also since $Y=U_y$ and $X=U_x$ joint normality of $U_y$ and $U_x$ means joint normality of $X,Y$.

Back to the original question. Let me first add a few qualifiers on this in terms of what we are assuming

1. $Y=\beta_0+\beta_1X+\varepsilon$ where $E(\varepsilon)=0$ is a correct model, meaning $E(Y|X)=\beta_0+\beta_1X$.
2. We are not conditioning on anything else (including something that might be specific to selection, aka selection bias)
3. $X$ and $\varepsilon$ are random variables that are both realized before $Y$.
4. the parameter we want to know if it is causal or not is $\frac{cov(X,Y)}{var(X)}$, which is what we estimate plugging in sample estimates of covariance and variance

Then the answer is yes, if $X$ and $\varepsilon$ are independent (in fact they only need to be uncorrelated) then $\frac{cov(X,Y)}{var(X)}$ is causal. It can easily be seen if we construct a partial (faithful) DAG that $X$ and $\varepsilon$ are independent if and only if there is no backdoor path from $X$ to $Y$ (which is the condition we need for the relationship between $X$ and $Y$ to be causal). Note that since $Y$ is a deterministic function of $X$ and $\varepsilon$ it means the only arrows that can point directly to $Y$ are ones that start from $X$ or $\varepsilon$. This part of the graph looks like $X\rightarrow Y \leftarrow \varepsilon$ (and there are no other incoming arrows to $Y$). This means that any backdoor path from $X$ to $Y$ must be intersected by $\varepsilon$ and that would make $X$ and $\varepsilon$ dependent. Note that $Y$ is a collider between $X$ and $\varepsilon$ which blocks the path from $X$ to $\varepsilon$ that is intersected by $Y$.