Regression and causality in econometrics

Question

In regression in general and in linear regression in particular, causal interpretation of parameters is sometimes permitted. At least in econometrics literature, but not only, when causal interpretation is permitted is not so clear. For a discussion, you can see Regression and Causation: A Critical Examination of Six Econometrics Textbooks - Chen and Pearl (2013).

For proper handling of causality in a statistical model the best way probably is to use Structural Causal Model as explained, for example (shortly), in Trygve Haavelmo and the Emergence of Causal Calculus – Pearl 2012 Feb.

However, currently, these are not the standard method in basic econometrics model (classic multiple linear regression). Indeed is frequently used the concept of “true model” or “data generating process” that sometimes has explicit causal meaning. In any case, I want to consider only causal sense. Therefore if we estimate the sample counterpart of “true model” we achieve causal interpretation of parameters.

Keeping in mind the above consideration, my attempt is to grasp

• the link between the concept of “true model” (of current econometrics textbooks) and structural causal model (of Pearl) … if any.

• The link between the previous point and the concept of randomized controlled experiment, as used in laboratory, that sometimes is the reference point in econometrics observational study (as good as it). For example Stock and Watson (2013) spend a lot of discussion about that (particularly cap 13). Moreover, in Pearl 2012feb page 14, there is a debate review between “structuralists” and “experimentalists” that is strongly related to this point.

Can you explain to me something about these two points in the simplest possible scenario?

Answer 1

In the context of the Pearl paper you've given, what most econometricians would call a true model is input I-1 to the Structural Causal Model: a set of assumptions $A$ and a model $M_A$ that encodes these assumptions, written as a system of structural equations (as in Models 1 and 2) and a list of statistical assumptions relating the variables. In general, the true model need not be recursive, so the corresponding graph can have cycles.

What's an example of a true model? Consider the relationship between schooling and earnings, described in Angrist and Pischke (2009), section 3.2. For individual $i$, what econometricians would call the true model is an assumed function mapping any level of schooling $s$ to an outcome $y_{si}$: $$ y_{si} = f_i(s). $$ This is exactly the potential outcome. One could go further and assume a parametric functional form for $f_i(s)$. For example, the linear constant effects causal model: $$ f_i(s) = \alpha + \rho s + \eta_i. $$ Here, $\alpha$ and $\rho$ are unobserved parameters. By writing it this way, we assume that $\eta_i$ does not depend on $s$. In Pearl's language, this tells us what happens to expected earnings if we fix an individual's schooling at $s_i = s_0$, but we don't observe $\eta_i$: $$ E[y_{si} \mid do(s_i = s_0)] = E[f_i(s_0)] = \alpha + \rho s_0 + E[\eta_i]. $$ We haven't said what queries we're interested in, or what data we have. So the "true model" is not a full SCM. (This is generally true, not just in this example.)

What's the connection between a true model and a randomized experiment? Suppose an econometrician wants to estimate $\rho$. Just observing $(s_i, y_i)$ for a bunch of individuals isn't sufficient. This is identical to Pearl's point about statistical conditioning. Here $$ E[y_{si} \mid s_i = s_0] = E[f_i(s_0) \mid s_i = s_0] = \alpha + \rho s_0 + E[\eta_i \mid s_i = s_0]. $$ As Angrist and Pischke point out, $\eta_i$ may be correlated with $s_i$ in observational data, due to selection bias: an individual's decision about schooling might depend on her value of $\eta_i$.

Randomized experiments are one way to correct for this correlation. Using Pearl's notation loosely here, if we randomly assign our subjects to $do(s_i = s_0)$ and $do(s_i = s_1)$ then we can estimate $E[y_{si} \mid do(s_i = s_1)]$ and $E[y_{si} \mid do(s_i = s_0)]$. Then $\rho$ is given by: $$ E[y_{si} \mid do(s_i = s_1)] - E[y_{si} \mid do(s_i = s_0)] = \rho(s_1 - s_0). $$

With additional assumptions and data, there are other ways to correct for the correlation. A randomized experiment is only considered the "best" because we may not believe the other assumptions. For example, with the Conditional Independence Assumption and additional data, we could estimate $\rho$ by OLS; or we could bring in instrumental variables.

Edit 2 (CIA): This is mainly a philosophical point, and Angrist and Pischke may disagree with my presentation here. The Conditional Independence Assumption (selection on observables) lets us correct for selection bias. It adds an assumption about joint distributions: that $$ f_i(s) \perp\!\!\!\perp s_i \mid X_i $$ for all $s$. Using just conditional expectation algebra (see the derivation in Angrist and Pischke) it follows that we can write $$ y_i = f_i(s_i) = \alpha + \rho s_i + X_i' \gamma + v_i $$ with $E[v_i \mid X_i, s_i] = 0$. This equation allows us to estimate $\rho$ in the data using OLS.

Neither randomization nor the CIA goes into the system of equations that defines the true model. They are statistical assumptions that give us ways to estimate parameters of a model we've already defined, using the data we have. Econometricians wouldn't typically consider the CIA part of the true model, but Pearl would include it in $A$.

Answer 2

I will start with the second part of your question, which pertains to the difference between randomized control studies and observational studies, and will wrap it up with the part of your question pertaining to "true model" vs. "structural causal model".

I will use one of Pearl's examples, which is an easy one to grasp. You notice that when the ice cream sales are highest (in the summer), the crime rate is highest (in the summer), and when the ice cream sales are lowest (in the winter), the crime rate is lowest. This makes you wonder whether the level of ice cream sales is CAUSING the level of crime.

If you could perform a randomized control experiment, you would take many days, suppose 100 days, and on each of these days randomly assign the level of sales of ice cream. Key to this randomization, given the causal structure depicted in the graph below, is that the assignment of the level of ice cream sales is independent of the level of temperature. If such a hypothetical experiment could be performed you should find that on the days when the sales were randomly assigned to be high, the average crime rate is not statistically different than on days when the sales were assigned to be low. If you had your hands on such data, you'd be all set. Most of us, however, have to work with observational data, where randomization did not do the magic it did in the above example. Crucially, in observational data, we do not know whether the level of Ice Cream Sales was determined independently of Temperature or whether it depends on temperature. As a result, we'd have to somehow untangle the causal effect from the merely correlative.

Pearl's claim is that statistics doesn't have a way of representing E[Y|We Set X to equal a particular value], as opposed to E[Y|Conditioning on the values of X as given by the joint distribution of X and Y]. This is why he uses the notation E[Y|do(X=x)] to refer to the expectation of Y, when we intervene on X and set its value equal to x, as opposed to E[Y|X=x], which refers to conditioning on the value of X, and taking it as given.

What exactly does it mean to intervene on variable X or to set X equal to a particular value? And how is it different than conditioning on the value of X?

Intervention is best explained with the graph below, in which Temperature has a causal effect on both Ice Cream Sales and Crime Rate, and Ice Cream Sales has a causal effect on Crime Rate, and the U variables stand for unmeasured factors that affect the variables but we do not care to model these factors. Our interest is in the causal effect of Ice Cream Sales on Crime Rate and suppose that our causal depiction is accurate and complete. See the graph below.

Now suppose that we could set the level of ice cream sales very high and observe whether that would be translated into higher crime rates. To do so we would intervene on Ice Cream Sales, meaning that we do not allow Ice Cream Sales to naturally respond to Temperature, in fact this amounts to us performing what Pearl calls "surgery" on the graph by removing all the edges directed into that variable. In our case, since we're intervening on Ice Cream Sales, we would remove the edge from Temperature to Ice Cream sales, as depicted below. We set the level of Ice Cream Sales to whatever we want, rather than allow it be determined by Temperature. Then imagine that we performed two such experiments, one in which we intervened and set the level of ice cream sales very high and one in which we intervened and set the level of ice cream sales very low, and then observe how Crime Rate responds in each case. Then we'll start to get a sense of whether there is a causal effect between Ice Cream Sales and Crime Rate or not.

Pearl distinguished between intervention and conditioning. Conditioning here refers merely to a filtering of a dataset. Think of conditioning on Temperature as looking in our observational dataset only at cases when the Temperature was the same. Conditioning does not always give us the causal effect we're looking for (it doesn't give us the causal effect most of the time). It happens that conditioning would give us the causal effect in the simplistic picture drawn above, but we can easily modify the graph to illustrate an example when conditioning on Temperature would not give us the causal effect, whereas intervening on Ice Cream Sales would. Imagine that there is another variable which causes Ice Cream Sales, call it Variable X. In the graph is would be represented with an arrow into Ice Cream Sales. In that case, conditioning on Temperature would not give us the causal effect of Ice Cream Sales on Crime Rate because it would leave untouched the path: Variable X -> Ice Cream Sales -> Crime Rate. In contrast, intervening on Ice Cream Sales would, by definition, mean that we remove all arrows into Ice Cream, and that would give us the causal effect of Ice Cream Sales on Crime Rate.

I will just mention that one Pearl's greatest contributions, in my opinion, is the concept of colliders and how conditioning on colliders will cause independent variables to be likely dependent.

Pearl would call a model with causal coefficients (direct effect) as given by E[Y|do(X=x)] the structural causal model. And regressions in which the coefficients are given by E[Y|X] is what he says authors mistakenly call "true model", mistakenly that is, when they are looking to estimate the causal effect of X on Y and not merely to forecast Y.

So, what's the link between the structural models and what we can do empirically? Suppose you wanted to understand the causal effect of variable A on variable B. Pearl suggests 2 ways to do so: Backdoor criterion and Front-door criterion. I will expand on the former.

Backdoor Criterion: First, you need to correctly map out all the causes of each variable and using the Backdoor criterion identify the set of variables that you'd need to condition on (and just as importantly the set of variables you need to make sure you do not condition on - i.e. colliders) in order to isolate the causal effect of A on B. As Pearl points out, this is testable. You can test whether or not you've correctly mapped out the causal model. In practice, this is easier said than done and in my opinion the biggest challenge with Pearl's Backdoor criterion. Second, run the regression, as usual. Now you know what to condition on. The coefficients you'll get would be the direct effects, as mapped out in your causal map. Note that this approach is fundamentally different from the traditional approach used in estimating causality in econometrics - Instrumental Variable regressions.

Answer 3

The use of ‘causal’ in such regression/correlation based approaches is misleading, in my opinion. Path analysis, structural equation modeling, Granger causality, etc attempts to license causal inferences by imposing some fairly tenuous assumptions. In the case of Structural equation modeling for instance, the paths are directional and A appears to ‘cause’ B, but this simply means that the model as structured is ‘plausible’ in that it reproduces an observed covariance matrix (in fact, the direction of the paths don’t even matter much - just the constraints).