55
$\begingroup$

In the year 2000, Judea Pearl published Causality. What controversies surround this work? What are its major criticisms?

$\endgroup$
  • 10
    $\begingroup$ There's an informative discussion in the archives of Andrew Gelman's blog, including contributions from Pearl and other experts. $\endgroup$ – guest Apr 14 '12 at 4:01
  • 11
    $\begingroup$ Gelman discusses Pearl's Causality, in addition to SL Morgan and C Winship's Counterfactuals and Causal Models and A Sloman's Causal Models in a 2011 review essay in the Am. J. of Sociology. He is generally very supportive of Pearl's contributions, especially Pearl's formalization of causal models in terms of interventions (do-calculus). However, he remains concerned that state-of-the-art causal theory may still invite oversimplified causal models and subsequently false causal inferences from observational data. $\endgroup$ – jthetzel Aug 7 '12 at 15:06
  • 1
    $\begingroup$ @jthetzel: Thanks, that looks like a good answer to me. Would you mind adding it? $\endgroup$ – Neil G Aug 8 '12 at 19:17
35
$\begingroup$

Some authors dislike Pearl's focus on the directed acyclic graph (DAG) as the way in which to view causality. Pearl essentially argues that any causal system can be considered as a non-parametric structural equation model (NPSEM), in which the value of each node is taken as a function of its parents and some individual error term; the error terms between different nodes may in general be correlated, to represent common causes.

Cartwright's book Hunting Causes and Using Them, for example, gives an example involving a car engine, which she claims cannot be modelled in the NPSEM framework. Pearl disputes this in his review of Cartwright's book.

Others caution that the use of DAGs can be misleading, in that the arrows lend an apparent authority to a chosen model as having causal implications, when this may not be the case at all. See Dawid's Beware of the DAG. For example, the three DAGs $A \rightarrow B \rightarrow C$, $A \leftarrow B \rightarrow C$ and $A \leftarrow B \leftarrow C$ all induce the same probabilistic model under Pearl's d-separation criterion, which is that A is independent of C given B. They are therefore indistinguishable based upon observational data.

However they have quite different causal interpretations, so if we wish to learn about the causal relationships here we would need more than simply observational data, whether that be the results of interventional experiments, prior information about the system, or something else.

$\endgroup$
  • 1
    $\begingroup$ To be fair - far from being unaware of the three DAGs with the same probabilistic model, Pearl has been one of the chief promoters of the distinction between merely statistical-probabilistic-associational models and fully causal models. See for example Section 2 of ftp.cs.ucla.edu/pub/stat_ser/r354-corrected-reprint.pdf $\endgroup$ – Paul Dec 11 '17 at 18:02
  • $\begingroup$ @Paul yes indeed; I was just reporting other people's misgivings about using DAGs. I have no such misgivings - please edit if you think the reply is unfair! $\endgroup$ – rje42 Dec 13 '17 at 9:16
  • 4
    $\begingroup$ It just sounds like the message was totally lost in translation. Which is not necessarily the fault of your answer, if you're just reporting the criticisms people have made. The whole point of Pearl's work is that different causal models can generate the same probabilistic model and hence the same-looking data. So it's not enough to have a probabilistic model, you have to base your analysis and causal interpretation on the full DAG to get reliable results. If you're just reporting what people say, I don't think your answer needs editing, these comments are sufficient clarification. $\endgroup$ – Paul Dec 13 '17 at 16:58
  • $\begingroup$ voted +1 by the way. $\endgroup$ – Paul Dec 15 '17 at 20:23
16
$\begingroup$

I think this framework has a lot of trouble with general equilibrium effects or Stable Unit Treatment Value Assumption violations. In that case, the "untreated" observations no longer provide the desired counterfactual in a meaningful way. Massive job training programs that shift the entire wage distribution are one example. The counterfactual may not even be well-defined in some cases. In Morgan and Winship's Counterfactuals and Causal Models, they give an example of the claim that the 2000 election would have gone in favor of Al Gore if felons and ex-felons had been allowed to vote. They point out that the counterfactual world would have very different candidates and issues, so that you cannot characterize the alternative causal state. The ceteris paribus effect would not be the policy relevant parameter here.

$\endgroup$
  • $\begingroup$ It sounds like you're saying that some counterfactuals are not reasonable because it's not reasonable to assume that only one thing changes? In the felon example, the simple fact of felons being able to vote would imply many other differences between that potential world and our actual world, so it's not reasonable to change "just one thing"? $\endgroup$ – Paul Dec 11 '17 at 17:59
  • 2
    $\begingroup$ @Paul Yes, the "all else equal" cannot hold. $\endgroup$ – Dimitriy V. Masterov Dec 11 '17 at 22:45
  • 1
    $\begingroup$ Thanks. I think this is a fairly profound and underappreciated point about counterfactuals. People usually assume they can do whatever they want. But just like the real world, I guess the space of valid counterfactuals can have "multicollinearity". $\endgroup$ – Paul Dec 11 '17 at 23:03
0
$\begingroup$

The most important criticism of Pearl's system is, from my perspective, that it has not yielded any practical, empirical advances anywhere it has been used. Given how long it has been around, there's no reason to think it will ever be a practical tool. This indicates that it can be used for some theoretical and perhaps didactic purposes, but a practical researcher will gain little from studying it.

$\endgroup$
  • 1
    $\begingroup$ A warm welcome to this site, but your answer is totally ridiculous. $\endgroup$ – Neil G May 22 at 8:00
  • 1
    $\begingroup$ Why is it ridiculous? If Pearl promoted his system simply as some kind of conceptual, philosophical tool for understanding what causality is, I wouldn't have a problem with it. But he constantly talks it up as a "revolutionary" practical tool for researchers to use, which is simply bullshit. For example, in his latest book Pearl says that he "would not be surprised" if the front-door method "eventually becomes a serious competitor to randomized controlled trials", which is a strong claim given that there's not a single example of the method being used to solve any real problem, ever. $\endgroup$ – Matt May 22 at 8:27
  • 1
    $\begingroup$ It's ridiculous because his work has been cited tens of thousands of times. The front-door method was famously used to support the link between smoking and cancer in defiance of Ronald Fisher's testimony! $\endgroup$ – Neil G May 22 at 8:47
  • 5
    $\begingroup$ What does Pearl's citation count have to do with anything? My criticism is that the practical benefits that he has promised for decades have not materialized. Pearl came up with the front-door criterion decades after Fisher had died and the cancer-and-smoking controversy had settled down. How could the criterion have been used against Fisher? $\endgroup$ – Matt May 22 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.