Is causal inference only from data possible?

Question

Suppose we are given a dataset but not the capability of performing some AB testing. We do some regression using X as predictor and Y as response and get a model. Can we actually say something about the causal relationship between X and Y? Or is it simply impossible to say anything about the causal relationship at all?

For example, suppose the data we have is simply the fathers' height and the sons' height, and also suppose mother's height has no influence on son's height. We can get a good linear relationship using sons's height as X and fathers' height as Y. However, we cannot say that lower sons' height causes fathers' height to be lower.

In other words, I feel the causal inference has to eventually resort to some physical/mechanical mechanism instead of just by looking at the data. Am I missing something here?

Answer 1

Suppose we are given a dataset but not the capability of performing some AB testing. We do some regression using X as predictor and Y as response and get a model. Can we actually say something about the causal relationship between X and Y?

No you can't, even when all variables are observed, see here for instance. If you are given only distributional information about the data (ie., you know the joint distribution of the observed variables), but no information about how the data was generated (a causal model), causal inference is impossible. In short, you need causal assumptions to get causal conclusions. You can get started on learning causal inference with the references here.

It is easy to understand why that is the case by constructing an example where different causal models entail the same observed joint probability distribution. Consider that you have observed the joint probability distribution $P(x, y)$ of two random variables. Here, imagine you have no sampling uncertainty---so you have perfect knowledge of $P(x, y)$, which entails perfect knowledge of the regression function and so on. To simplify things, consider that, in your data, $P(x,y)$ was found to be jointly normal with mean $0$, variance 1 and covariance $\sigma_{xy}$ (this is without loss of generality, you can always standardize the data). What can you say about the causal effect of $x$ on $y$ or vice-versa?

With only this information, nothing. The reason here is that there are several causal models that would create the same observed distribution, yet have different interventional (and counterfactual) distributions. Here I will show three of such models. Notice all of them gives you the same observed $\sigma_{xy}$, but their causal conclusions are different: in the first model $X$ causes $Y$, in the second model $Y$ causes $X$, and, in the third model, neither causes each other --- $X$ and $Y$ are both common causes of the unobserved variable $Z$.

Model 1

$$ X = u_{x}\\ Y= \sigma_{yx}x + u_{y} $$

Where $U_{x} \sim \mathcal{N}(0, 1)$ and $U_{y} = \mathcal{N}(0, 1 - \sigma_{xy}^2)$.

Model 2

$$ Y = u_{y}\\ X = \sigma_{yx}y + u_{x} $$

Where $U_{x} \sim \mathcal{N}(0, 1 - \sigma_{xy}^2)$ and $U_{y} = \mathcal{N}(0, 1)$.

Model 3

$$ Z = U_{z}\\ X = \alpha Z + U_{x}\\ Y = \beta Z + U_{y} $$

Where $\alpha\beta= \sigma_{xy}$, $U_{z} = \mathcal{N}(0, 1)$, $U_{x} = \mathcal{N(0, 1- \alpha^2)}$ and $U_{y} = \mathcal{N(0, 1- \beta^2)}$.

Answer 2

From data alone, it's impossible. There could always be some factor outside the model that could influence both $X$ and $Y$ (or one of them). It's impossible to control for literally everything.

The closest we have is a randomized control experiment, but even that has problems with external validity (e.g. we assume that what happened and the conditions during the time of the experiment will persist into the indefinite future).

There is 'Granger causality' (which is not true causality), which basically says if the parameters on the lagged $X$ variables in a regression of $Y(t)$ on $X(t-1), ..., X(t-m), Y(t-1), ..., Y(t-m)$ are jointly significant, then $X$ 'Granger causes' $Y$. See Granger (1969).

Answer 3

Potentially. Your intuition about the necessity to "resort to some physical/mechanical mechanism" is correct but that does not mean that the explicit definition of such mechanism is required. We can relax this problem.

There is a lot of work on causal inference from observational data where we do not explicitly formulate the causal model in form of a clear parametric equation. There are "ML-flavoured" approaches like: "Learning Representations for Counterfactual Inference" by Johansson et al., "Causal inference by using invariant prediction" by Peters et al., "Causal Forests" by Athey and various collaborators that make significant inroads. Let's be clear: these approaches require substantial amounts of data and are far from prime-time ready. Nevertheless they offer evidence that while using observational data to answer causal questions is risky, obtaining answers is not impossible.

Final note: we have only recently have started coming up with "causal datasets" - datasets, where we have carefully annotated causal effects. The grand revolution in Computer Vision came through the abundance of available label training data. Causal inference work so far is not enjoying such a data-rich environment to work. Initiatives like the Causality workbench, the Causal Inference challenges, the Tubingen datasets observational samples give us test-beds that were simply unavailable only 10 years ago.