8
$\begingroup$

Teasing out the causal effect of one variable/treatment on another/outcome by blocking all the Backdoor Paths between treatment and outcome in the corresponding DAG (Directed Acyclic Graph) requires drawing a correct DAG in the first place. But can we ever be sure our DAG is correct?!

Can you point to a convincing/rigorous/commonly agreed to be correct causal study which estimated the causal effect by drawing a DAG and blocking all backdoor paths? If you know of such a study, why do you believe the DAG to be correct?

I've been intrigued by causal analysis using DAGs and backdoor paths but I do not read any academic journals so it is difficult for me to assess whether this technique is merely an interesting logical/theoretical setup or is actually practical/useful.

$\endgroup$
4
  • 2
    $\begingroup$ As far as I'm aware, the usual attitude is not "our DAG is absolutely correct", but "we assume that this DAG applies and based on that, we adjust for variables x y z to get unbiased estimates". One giant advantage of using DAGs is that they make the (usually unstated) assumptions about the causal relationships between variables explicit. It's quite possible that researchers criticize the stipulated DAG of other researchers. $\endgroup$ – COOLSerdash Jan 18 '20 at 16:38
  • $\begingroup$ Thank you for that added color. Perhaps you know of a convincing study that estimated the causal effect in 2 ways: 1) with a DAG and blocking backdoor paths (which often translates into requiring that most of the DAG be correct) and 2) another method (perhaps one that requires only a very small part of the DAG to be correct)? Are serious academic journals accepting papers on simple faith that the DAG sounds credible? $\endgroup$ – ColorStatistics Jan 18 '20 at 17:00
  • 1
    $\begingroup$ The backdoor path criterion is a formal way about how to reason about whether a set of variables is sufficient so that if you condition on them, the association between $X$ and $Y$ reflects how $X$ affects $Y$ and nothing else. This strategy, adding control variables to a regression, is by far the most common in the empirical social sciences. It's everywhere and if the authors gave reasoning why their control variables are needed and sufficient, it will be special cases of the reasoning formalised in the backdoor criterion. $\endgroup$ – CloseToC Jan 18 '20 at 19:05
  • $\begingroup$ So it sounds like it is commonly used in some social sciences. Curiously, I haven't seen the method described in any Econometrics book. $\endgroup$ – ColorStatistics Jan 18 '20 at 22:53
6
$\begingroup$

No, we can never be sure that the DAG is correct. This is one of the fundamental principles of causal inference informed by DAGs. DAGs are a non-parametric abstraction of reality. You will find in much of the DAG literature things like:

In causal diagrams, an arrow represents a "direct effect" of the parent on the child, although this effect is direct only relative to a certain level of abstraction, in that the graph omits any variables that might mediate the effect represented by the arrow.

Greenland and Pearl, 2017

This is completely unavoidable. Take pharmacological research. There are many, many cases of drugs which reach the market, where the researchers do not know the actual biological mechanism that causes their product to work. They may have theories, and these theories can be encapsulated using DAGs. The resulting analysis is conditional on the DAG being correct (at a level of abstraction). Other researchers may have different theories and consequently different DAGs, and that is completely OK.

$\endgroup$
3
  • $\begingroup$ Thank you, Robert. I am a bit surprised that more is not done to convince the reader that this "abstraction of reality" is credible. Your point regarding the fact that oftentimes "the researchers do not know the actual biological mechanism that causes their product to work" is a good one and understood. $\endgroup$ – ColorStatistics Jan 19 '20 at 15:39
  • $\begingroup$ Pearl in his primer book (page 50) expresses his excitement about the fact that "... it allows us to search a data set for the causal model that could have generated it", where "it" refers to "... we could test and reject many possible models in this way... whittling down the set of possible models [DAGs] to only a few whose testable implications do not contradict the dependencies present in the data set". How can we not be concerned with over-fitting in any DAG generated in this way? So either we have to accept it on faith or be really concerned about over-fitting? $\endgroup$ – ColorStatistics Jan 19 '20 at 15:41
  • $\begingroup$ You are welcome. Yes, I agree that such a procedure could be liable to over-fitting and it is not something I would recommend. The DAG should be the starting point, informed by expert domain knowledge. Having several DAGs shouldn't be a problem if there are competing theories about how the data are generated, and It might be an interesting theoretical exercise to do some kind of search based on all possible DAGs and finally choosing the "best" one, but that sounds very dangerous to me in an applied research setting. Do remember that Pearl comes from a theoretical background though. $\endgroup$ – Robert Long Jan 19 '20 at 15:51
2
$\begingroup$

We can first think more generally about what a causal diagram really is. Then let's discuss how one might practically use them as an informative prior, and jointly with observational data, to confidently predict causal effects.

A causal diagram is a directed acyclic graph (DAG) representation of the functional relationships between the variables (i.e. nodes) within the distribution. And the structure of the graph serves to encode the conditional dependence or independence among the variables. The diagram essentially asserts our assumptions about the world in a easy-to-understand visual format. Provided with a joint distribution p(a,b,c), the same distribution can be written as either:

formula

formula

So which causal diagram is the correct one for the joint distribution? The example demonstrates that the mapping of causal diagrams to our observational data is many to one. Multiple correct hypothesis are plausible, and it is usually impossible to definitely choose between them just by looking at the observational data only.

How can we then use observational data to infer the correct diagram? For a said causal diagram, we mimic the effects of a intervention by conditioning on a variable (i.e. we force it to take a particular value). The action is encapsulated by the do-operator in p(Y|do(X)) and more formally by do-calculas, a tool for causal inference that allows us to disambiguate what needs to be estimated from the observational data. The front- and back-door approaches are but just two doors through which we can eliminate all the do's in our quest to climb Mount Intervention.

Suffice to say, by removing all incoming edges to the node of interest, an intervention modifies the original joint distribution to become the post-interventional distribution. A causal query becomes identifiable if we can remove all do-operators and therefore we can use the observational data to estimate causal effect. Else the causal query is considered non-identifiable and a real-world interventional experiment would be required for determining the causal effect.

While a researcher may never be completely persuaded in the soundness and integrity of the causal diagram they've constructed, they do have mechanisms in place to empirically test a partial collection of relationships between the sets of variables. If the dependencies and independencies are not present in the observational data, this might be a signal that the diagram is inaccurate. The researcher can then iteratively test and update the causal diagram to be more inline with the information contained within the observational data (and domain knowledge if applicable).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.