7
$\begingroup$

In regression in general and in linear regression in particular causal interpretation about parameters is sometimes permitted. At least in econometrics literature, but not only, when causal interpretation is permitted is not so clear; for a discussion you can see: Regression and Causation: A Critical Examination of Six Econometrics Textbooks - Chen and Pearl (2013).

For a proper handling of causality in statistical model the best way probably is to use Structural Causal Model as explained, for example (shortly), in: Trygve Haavelmo and the Emergence of Causal Calculus – Pearl 2012 feb.

However, currently, these are not the standard method in basics econometrics model (classic multiple linear regression). Indeed is frequently used the concept of “true model” or “data generating process” that sometime have explicit causal meaning. In any case I want to consider only causal sense. Therefore if we estimate the sample counterpart of “true model” we achieve causal interpretation about parameters.

Keeping in mind the above consideration, my attempt is to grasp

  • the link between the concept of “true model” (of current econometrics textbooks) and structural causal model (of Pearl) … if any.

  • The link between the previous point and the concept of randomized controlled experiment, as used in laboratory, that sometimes is the reference point in econometrics observational study (as good as it). For example Stock and Watson (2013) spend a lot of discussion about that (particularly cap 13). Moreover in Pearl 2012feb pag 14 there is a debate review between “structuralists” and “experimentalists” that is strongly related with this point.

Can you explain me something about these two point in the simplest as possible scenario?

$\endgroup$
  • 2
    $\begingroup$ There are multiple schools of thought about causality in econometrics. See e.g., Heckman and Pinto (2015) commenting on Pearl, or the applications of the Rubin potential outcomes model (for example in Mostly Harmless Econometrics by Angrist and Pischke). $\endgroup$ – Frank Nov 14 '18 at 18:25
  • $\begingroup$ there are no "true" models in economtrics $\endgroup$ – Aksakal Nov 14 '18 at 22:31
  • 1
    $\begingroup$ @Frank: thank you for the paper suggested, they will be surely useful. Regarding the Angrist and Pischke book, I read it. I surveyed about ten econometrics textbooks and Mostly Harmless Econometrics represent the best about causality (my opinion but probably not only). I learned a lot from that. However, at least in my opinion, even this book is exhaustive. For example it says nothing about structural causal model and link between it and potential outcome language. Finally it does not offer an answer a my two question above. Or, at least, I "don't see" it. $\endgroup$ – markowitz Nov 15 '18 at 8:36
  • 1
    $\begingroup$ @Aksakal: answer like your says both, everything or nothing. In my opinion says nothing because it don't says me nothing about: firstly, why in many econometric books "true model" is used (and seems very useful if not essential); what are the links among your answer and my questions? If links do not exist the presence of "true model" became very hard to explain. In any case I known that "true model" is sometimes criticized concept and that "real world" is something different and links between them can be artificial, but this is the theory. My question is about econometric theory. $\endgroup$ – markowitz Nov 15 '18 at 8:52
  • $\begingroup$ @markowitz: Angrist and Pischke do discuss this, but their language might be confusing you. See section 3.2.1: "the function $f_i(s)$ tells us what $i$ would earn for any value of schooling, $s$. In other words, $f_i(s)$ answers causal 'what if' questions." See also the discussion of the linear constant effects causal model (at the end of the section). $\endgroup$ – Frank Nov 15 '18 at 14:01
4
$\begingroup$

In the context of the Pearl paper you've given, what most econometricians would call a true model is input I-1 to the Structural Causal Model: a set of assumptions $A$ and a model $M_A$ that encodes these assumptions, written as a system of structural equations (as in Models 1 and 2) and a list of statistical assumptions relating the variables. In general, the true model need not be recursive, so the corresponding graph can have cycles.

What's an example of a true model? Consider the relationship between schooling and earnings, described in Angrist and Pischke (2009), section 3.2. For individual $i$, what econometricians would call the true model is an assumed function mapping any level of schooling $s$ to an outcome $y_{si}$: $$ y_{si} = f_i(s). $$ This is exactly the potential outcome. One could go further and assume a parametric functional form for $f_i(s)$. For example, the linear constant effects causal model: $$ f_i(s) = \alpha + \rho s + \eta_i. $$ Here, $\alpha$ and $\rho$ are unobserved parameters. By writing it this way, we assume that $\eta_i$ does not depend on $s$. In Pearl's language, this tells us what happens to expected earnings if we fix an individual's schooling at $s_i = s_0$, but we don't observe $\eta_i$: $$ E[y_{si} \mid do(s_i = s_0)] = E[f_i(s_0)] = \alpha + \rho s_0 + E[\eta_i]. $$ We haven't said what queries we're interested in, or what data we have. So the "true model" is not a full SCM. (This is generally true, not just in this example.)

What's the connection between a true model and a randomized experiment? Suppose an econometrician wants to estimate $\rho$. Just observing $(s_i, y_i)$ for a bunch of individuals isn't sufficient. This is identical to Pearl's point about statistical conditioning. Here $$ E[y_{si} \mid s_i = s_0] = E[f_i(s_0) \mid s_i = s_0] = \alpha + \rho s_0 + E[\eta_i \mid s_i = s_0]. $$ As Angrist and Pischke point out, $\eta_i$ may be correlated with $s_i$ in observational data, due to selection bias: an individual's decision about schooling might depend on her value of $\eta_i$.

Randomized experiments are one way to correct for this correlation. Using Pearl's notation loosely here, if we randomly assign our subjects to $do(s_i = s_0)$ and $do(s_i = s_1)$ then we can estimate $E[y_{si} \mid do(s_i = s_1)]$ and $E[y_{si} \mid do(s_i = s_0)]$. Then $\rho$ is given by: $$ E[y_{si} \mid do(s_i = s_1)] - E[y_{si} \mid do(s_i = s_0)] = \rho(s_1 - s_0). $$

With additional assumptions and data, there are other ways to correct for the correlation. A randomized experiment is only considered the "best" because we may not believe the other assumptions. For example, with the Conditional Independence Assumption and additional data, we could estimate $\rho$ by OLS; or we could bring in instrumental variables.

Edit 2 (CIA): This is mainly a philosophical point, and Angrist and Pischke may disagree with my presentation here. The Conditional Independence Assumption (selection on observables) lets us correct for selection bias. It adds an assumption about joint distributions: that $$ f_i(s) \perp\!\!\!\perp s_i \mid X_i $$ for all $s$. Using just conditional expectation algebra (see the derivation in Angrist and Pischke) it follows that we can write $$ y_i = f_i(s_i) = \alpha + \rho s_i + X_i' \gamma + v_i $$ with $E[v_i \mid X_i, s_i] = 0$. This equation allows us to estimate $\rho$ in the data using OLS.

Neither randomization nor the CIA goes into the system of equations that defines the true model. They are statistical assumptions that give us ways to estimate parameters of a model we've already defined, using the data we have. Econometricians wouldn't typically consider the CIA part of the true model, but Pearl would include it in $A$.

$\endgroup$
  • $\begingroup$ I already see the Angrist and Pischke example and I thought about it. That was the best example/explanation that I never found. Your addition represent what I was looking for. Thank you. $\endgroup$ – markowitz Nov 15 '18 at 22:31
  • $\begingroup$ However let me add some points and specific questions. Angrist and Pischke call the model as “linear constat effect causal model” but my doubt was exactly if that model was interpretable as “true model” that I usually see in many econometrics textbooks (for simple example see my comment on ColorStatistics answer). My favourite answer was no, because in the usual “true model” is explicitly valid some form of exogeneity that is indeed precisely violated in the present case. This is exactly your final consideration about my phrase but I was aware of this problem. $\endgroup$ – markowitz Nov 15 '18 at 22:31
  • $\begingroup$ Now, Angrist and Pischke never speak about “true model” but you suggest this interpretation for it. Probably you are right. In any case you write: “So the "true model" is just one part of the SCM.”. Ok! Your is a well posed form of one guess that I already had in mind. This conclusion hold even in the case of the “true model” that I was remember before? $\endgroup$ – markowitz Nov 15 '18 at 22:31
  • $\begingroup$ In the Angrist and Pischke's exposition the CIA plays a key role, and actually play a key role in causality. This is essential for eliminating selection bias. However in the example the causal assumptions explicitly preceded the CIA intervention. I think that if we take the general model $y = a + bx + cZ + e$, where $Z$ is a set of controls, the CIA condition (on $Z$) is never enough. Causal assumption must precede $y$,$x$ relationship shown in the equation. It’s correct? I think yes otherwise the saying “no causal assumpions no causal conclusions” is violated. $\endgroup$ – markowitz Nov 15 '18 at 22:32
  • 1
    $\begingroup$ @Frank you can give any partial specification of a structural causal model, the set of assumptions $A$ that Pearl refers to. Traditionally, this will be qualitative statements of the functional relationships that you can defend on scientific grounds. The CIA is a partial specification of an SCM as well, you are asserting that the potential response $Y_{s}$ does not depend on $S$ given a set of covariates, this imposes constraints on the possible systems of equations. $\endgroup$ – Carlos Cinelli Nov 16 '18 at 22:07
5
$\begingroup$

I will start with the second part of your question, which pertains to the difference between randomized control studies and observational studies, and will wrap it up with the part of your question pertaining to "true model" vs. "structural causal model".

I will use one of Pearl's examples, which is an easy one to grasp. You notice that when the ice cream sales are highest (in the summer), the crime rate is highest (in the summer), and when the ice cream sales are lowest (in the winter), the crime rate is lowest. This makes you wonder whether the level of ice cream sales is CAUSING the level of crime.

If you could perform a randomized control experiment, you would take many days, suppose 100 days, and on each of these days randomly assign the level of sales of ice cream. Key to this randomization, given the causal structure depicted in the graph below, is that the assignment of the level of ice cream sales is independent of the level of temperature. If such a hypothetical experiment could be performed you should find that on the days when the sales were randomly assigned to be high, the average crime rate is not statistically different than on days when the sales were assigned to be low. If you had your hands on such data, you'd be all set. Most of us, however, have to work with observational data, where randomization did not do the magic it did in the above example. Crucially, in observational data, we do not know whether the level of Ice Cream Sales was determined independently of Temperature or whether it depends on temperature. As a result, we'd have to somehow untangle the causal effect from the merely correlative.

Pearl's claim is that statistics doesn't have a way of representing E[Y|We Set X to equal a particular value], as opposed to E[Y|Conditioning on the values of X as given by the joint distribution of X and Y]. This is why he uses the notation E[Y|do(X=x)] to refer to the expectation of Y, when we intervene on X and set its value equal to x, as opposed to E[Y|X=x], which refers to conditioning on the value of X, and taking it as given.

What exactly does it mean to intervene on variable X or to set X equal to a particular value? And how is it different than conditioning on the value of X?

Intervention is best explained with the graph below, in which Temperature has a causal effect on both Ice Cream Sales and Crime Rate, and Ice Cream Sales has a causal effect on Crime Rate, and the U variables stand for unmeasured factors that affect the variables but we do not care to model these factors. Our interest is in the causal effect of Ice Cream Sales on Crime Rate and suppose that our causal depiction is accurate and complete. See the graph below.

enter image description here

Now suppose that we could set the level of ice cream sales very high and observe whether that would be translated into higher crime rates. To do so we would intervene on Ice Cream Sales, meaning that we do not allow Ice Cream Sales to naturally respond to Temperature, in fact this amounts to us performing what Pearl calls "surgery" on the graph by removing all the edges directed into that variable. In our case, since we're intervening on Ice Cream Sales, we would remove the edge from Temperature to Ice Cream sales, as depicted below. We set the level of Ice Cream Sales to whatever we want, rather than allow it be determined by Temperature. Then imagine that we performed two such experiments, one in which we intervened and set the level of ice cream sales very high and one in which we intervened and set the level of ice cream sales very low, and then observe how Crime Rate responds in each case. Then we'll start to get a sense of whether there is a causal effect between Ice Cream Sales and Crime Rate or not.

enter image description here

Pearl distinguished between intervention and conditioning. Conditioning here refers merely to a filtering of a dataset. Think of conditioning on Temperature as looking in our observational dataset only at cases when the Temperature was the same. Conditioning does not always give us the causal effect we're looking for (it doesn't give us the causal effect most of the time). It happens that conditioning would give us the causal effect in the simplistic picture drawn above, but we can easily modify the graph to illustrate an example when conditioning on Temperature would not give us the causal effect, whereas intervening on Ice Cream Sales would. Imagine that there is another variable which causes Ice Cream Sales, call it Variable X. In the graph is would be represented with an arrow into Ice Cream Sales. In that case, conditioning on Temperature would not give us the causal effect of Ice Cream Sales on Crime Rate because it would leave untouched the path: Variable X -> Ice Cream Sales -> Crime Rate. In contrast, intervening on Ice Cream Sales would, by definition, mean that we remove all arrows into Ice Cream, and that would give us the causal effect of Ice Cream Sales on Crime Rate.

I will just mention that one Pearl's greatest contributions, in my opinion, is the concept of colliders and how conditioning on colliders will cause independent variables to be likely dependent.

Pearl would call a model with causal coefficients (direct effect) as given by E[Y|do(X=x)] the structural causal model. And regressions in which the coefficients are given by E[Y|X] is what he says authors mistakenly call "true model", mistakenly that is, when they are looking to estimate the causal effect of X on Y and not merely to forecast Y.

So, what's the link between the structural models and what we can do empirically? Suppose you wanted to understand the causal effect of variable A on variable B. Pearl suggests 2 ways to do so: Backdoor criterion and Front-door criterion. I will expand on the former.

Backdoor Criterion: First, you need to correctly map out all the causes of each variable and using the Backdoor criterion identify the set of variables that you'd need to condition on (and just as importantly the set of variables you need to make sure you do not condition on - i.e. colliders) in order to isolate the causal effect of A on B. As Pearl points out, this is testable. You can test whether or not you've correctly mapped out the causal model. In practice, this is easier said than done and in my opinion the biggest challenge with Pearl's Backdoor criterion. Second, run the regression, as usual. Now you know what to condition on. The coefficients you'll get would be the direct effects, as mapped out in your causal map. Note that this approach is fundamentally different from the traditional approach used in estimating causality in econometrics - Instrumental Variable regressions.

$\endgroup$
  • $\begingroup$ I understand your explanation but I fear that it does not give an answer about my question. Except, maybe, the final phrase but I partially disagree with it. You firstly explain the usefulness of an experiment. I agree with you about that. After you said that (shortly) in observational context we have to “somehow untangle the causal effect from the merely correlative” without experimental manipulations. I known. $\endgroup$ – markowitz Nov 15 '18 at 11:14
  • $\begingroup$ After, since the conceptual difference between conditioning and intervention, you introduce Pearl interventional notation’s. I understand the usefulness of interventional notation. I’m not confident with causal graph but surely it is useful too. Pearl’s contributions are important. Finally you said that: “Pearl would call a model with causal coefficients (direct effect) as given by $E[Y|do(X=x)]$ the structural causal model.”. Ok, I understand. $\endgroup$ – markowitz Nov 15 '18 at 11:15
  • 1
    $\begingroup$ “regressions in which the coefficients are given by $E[Y|X]$ is what he says authors mistakenly call "true model", mistakenly that is, when they are looking to estimate the causal effect of $X$ on $Y$ and not merely to forecast $Y$.” This seems me not strictly true because $E[Y|X]$ is not always the true model. Any OLS regression is a conditional expectation but not every of these are counterpart of the true model frequently used in econometric textbooks. $\endgroup$ – markowitz Nov 15 '18 at 11:15
  • $\begingroup$ Let me use your/Pearl example: $X1$= temperature, $X2$= ice cream quantity, $Y$= crime rate. It’s possible to define the true (causal) model as: $Y = beta_0 + beta_1 *X1 + beta_2 * X2 + u$ but we estimate the underspecified model $Y = alfa_0 + alfa_1 *X1 + e$. It’s possible demonstrate that alfas parameters are biased estimates of the true causal parameters (betas); however the latter remain a conditional expectation. This is omitted variables problem. Indeed if we estimate the correct specified form, or a “longer”, the estimates parm. are unbiased and its causal interpretation is permitted. $\endgroup$ – markowitz Nov 15 '18 at 11:16
  • $\begingroup$ The explanation, at least in simple case like above, can be complete also without Pearl’s (and related) contributions. Exactly for reasons like the above I looking for the relations among: Structural Causal Model (from Pearl), experimental language (from Rubin); usual true model used in its causal sense (from many econometrics textbooks). I’m convinced that links exist but I’m doubt about the form of them. $\endgroup$ – markowitz Nov 15 '18 at 11:17
0
$\begingroup$

The use of ‘causal’ in such regression/correlation based approaches is misleading, in my opinion. Path analysis, structural equation modeling, Granger causality, etc attempts to license causal inferences by imposing some fairly tenuous assumptions. In the case of Structural equation modeling for instance, the paths are directional and A appears to ‘cause’ B, but this simply means that the model as structured is ‘plausible’ in that it reproduces an observed covariance matrix (in fact, the direction of the paths don’t even matter much - just the constraints).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.