30
$\begingroup$

My understanding is that probability (at least from a frequentist viewpoint) is a mathematical tool for modeling correlations. So, for example, we can say that two events $X$ and $Y$ are defined to be independent if $P(X\cap Y) = P(X)P(Y)$, or equivalently $P(X|Y) = P(X)$, and so on. However, something like whether or not $P(X|Y) = P(X)$ tells us nothing about causation. This leads me to the question that this post is about.

Is there any sort of theory or mathematical field of study that concerns itself with modeling causation?

I suspect there are answers in two possible forms. The first is that there might be specialized models specific to a given domain of study (biology, physics, economics). The second might be some generalized abstract theory akin to constructor theory that honestly only a mathematician would think up.

An answer in any possible form would be appreciated.

Improve this question
$\endgroup$
6
  • 16
    $\begingroup$ Do you mean something like causal inference? $\endgroup$
    – Dave
    Commented Mar 18, 2022 at 15:11
  • 2
    $\begingroup$ I can't give a detailed answer as I'm not familiar with the field, but if you're not already familiar with it you might want to take a look at some of the books by Judea Pearl that deals explicitly with causality. For example Causality, Models Reasoning and Inference. $\endgroup$
    – DancingIceCream
    Commented Mar 18, 2022 at 15:11
  • 9
    $\begingroup$ I agree with both commenters. The field is known as causal inference and is an old, well-established, and highly fruitful field of statistics research. It is studied and applied in various disciplines, including economics, epidemiology, sociology, and psychology, each of which have slightly differing perspectives on the matter. Pearl's work is one of the most prominent formalizations of causal inference and it is highly worth a read. He has several books of varying technicality that can introduce you to the topic. $\endgroup$
    – Noah
    Commented Mar 18, 2022 at 15:39
  • 6
    $\begingroup$ For some context: establishing causation in the real world where you never have the full picture is basically the domain of science. There, causation technically can never be proved with absolute certainty, its semantics depend on an interpretative framework (theoretical model) within which you operate, and the methodology involved is ultimately about increasing/decreasing confidence in your model through comparison of prediction and experiment/observation, and about discriminating between alternatives on this basis. $\endgroup$
    – Filip Milovanović
    Commented Mar 19, 2022 at 15:23
  • 1
    $\begingroup$ You might also want to recognize that there are different analytic causal formalisms: structural causal models—e.g., Pearl, Imbens, etc., qualitative loop models—e.g., Levins, Dambacher, etc., empirical dynamic modeling—e.g., Sugihara &Co., quasi-formalisms—Granger, and informalisms—e.g., Bradford Hill. $\endgroup$
    – Alexis
    Commented Mar 17, 2023 at 16:42

4 Answers 4

Reset to default
40
$\begingroup$

There are two main approaches to the New Causal Revolution. One is the graphical approach (as in, directed acyclic graphs), championed by Judea Pearl. The other is the potential outcomes framework, championed by Donald Rubin.

For the graphical approach, I recommend these books in this order:

  1. The Book of Why, by Pearl and MacKenzie. Prerequisite: introductory statistics.
  2. Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell. Prerequisites: the full calculus sequence, followed by mathematical statistics.
  3. Causality: Models, Reasoning, and Inference, 2nd Ed., by Pearl. This book is extremely difficult (I have not read it, though I have skimmed in parts) but has almost everything in there. One recent development not in this book is the maximal ancestor graph approach. Prerequisites: first mathematical statistics, then Bayesian statistics (I would recommend first Bayesian Statistics for Beginners: a step-by-step approach and then Introduction to Bayesian Statistics, 3rd Ed.; and finally Bayesian networks). Pearl's book has accompanying homework sets on Pearl's webpage: go to CAUSALITY, then 7. Viewgraphs and homeworks for instructors. Indeed, Pearl's UCLA webpage has errata for all three of the above books.

For the potential outcomes framework, the main book appears to be Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction, by Rubin and Imbens. I have not read it; but the prerequisites appear to be mathematical statistics at least - analysis and design of experiments wouldn't hurt, nor would Bayesian statistics (see #3 above for recommendations there). One weakness of this book is that it has no exercises or errata page.

The two approaches have different strengths and weaknesses, but as noted by Carlos in his comment, they are theoretically unified via Structural Causal Modeling - addressed in several of the books above.

Improve this answer
$\endgroup$
2
  • 5
    $\begingroup$ Adrian, the two approaches, from the theoretical point of view, are the same, see here: stats.stackexchange.com/questions/559240/… $\endgroup$
    – Carlos Cinelli
    Commented Mar 19, 2022 at 3:47
  • 3
    $\begingroup$ @CarlosCinelli Right! Which is certainly what we would hope, rather like the Heisenberg vs. Schrodinger approach to quantum mechanics having been shown to be equivalent. They still have their different strengths and weaknesses. Definitely agree that structural equation modeling is the underlying connection between the two. I think the graphical approach has a slight advantage in that you can infer some things about your system even without the structural equations. I have not yet studied PO, so I can't say if a similar thing is possible there. $\endgroup$
    – Adrian Keister
    Commented Mar 19, 2022 at 16:05
22
$\begingroup$

Causal inference is a part of statistical inference, so it falls within the field of statistics. Causal inference generally requires inference of statistical associations under an appropriate experimental structure that limits statistical associations to certain structures. This is dealt with in specialist books that look at the interaction of causality and probability, most notably the excellent works of Judea Pearl (see e.g., Pearl 2009, Pearl 2015, and Pearl, Glymour and Jewell 2016) and the potential-outcome framework of Donald Rubin (see e.g., Holland 1986, Rubin 1991, Rubin 2005). Pearl has criticised the statistical profession for paying insufficient attention to this material (see related question here), but his works nevertheless fall within the field of statistics and they can properly be regarded as contributions in the interface of probability, causality, and statistics.

As you will see from reading these works, it is possible to augment traditional probability theory by adding an operator to represent an action/intervention in the system (called the "do" operator), which allows causality to be built into the analysis at an axiomatic level. This is a useful extension of traditional probability theory. Presently, the statistics curriculum for students does not incorporate much of this material, except in some specialist classes that occur late in the program. It is my hope that this extension to probability theory will eventually be built into the statistical curriculum in a more cohesive manner, so that students become fluent in causal reasoning earlier on in their statistical studies, rather than seeing it as an add-on that they encounter only later in their career.

Improve this answer
$\endgroup$
9
  • 1
    $\begingroup$ +1: And I couldn't agree more, Ben. It is time that statistics curricula incorporate causal inference into the curriculum. These days it seems to me that econometrics and public health have done so, with other programs still not having a single course on causal inference with observational data (it is somewhat more common to see a course in Experimental Design). $\endgroup$
    – ColorStatistics
    Commented Mar 19, 2022 at 2:19
  • 1
    $\begingroup$ 1/2 Ben, I must sadly disagree ('sadly', because I find your contributions around these parts to be joyously Rock Star with 99.9% confidence :). To me the issues are two-fold. First, the 'do operator', the counterfactual formal causal formalism of DAGs, 'structural causal models': these do not appear in statistics textbooks, or in the axioms of, e.g., probability theory. Second, statistics has no theory of causation—no philosophical articulation of what causation means. $\endgroup$
    – Alexis
    Commented Mar 21, 2022 at 16:29
  • 1
    $\begingroup$ 2/2 For sure, there's been over a century of statisticians (or at least their textbooks and articles) saying "correlation is not causation". But that is not a causal theory or formalism: that's an abdication of causal inference as outside of statistics. Pearl's point about scattershot causal paradoxes and problems lacking a unifying framework in statistics' history, which are formally, readily and patly mapped out with causal formalisms indicates a new formal science complimenting statistics. $\endgroup$
    – Alexis
    Commented Mar 21, 2022 at 16:33
  • 1
    $\begingroup$ As an analogy, prior to about the late 19th century/early 20th century, the whole foundations of mathematics was a bit scattershot and lacked a unifying framework. Then along came a bunch of logicians/mathematicians who supplied one. We didn't then say that mathematics has been around for thousands of years and we only just got this, therefore it is a new branch of science. We just considered this to be more mathematics to fill in those gaps. $\endgroup$
    – Ben
    Commented Mar 21, 2022 at 22:03
  • 1
    $\begingroup$ That's a great point. I legit think the formal causal calculus doesn't get far without statistics. :) $\endgroup$
    – Alexis
    Commented Mar 21, 2022 at 22:03
14
$\begingroup$

Adrian Keister provided a great answer. My answer continues his. It took me a while to realize that the 2 different approaches to causal inference (graphical approach and potential outcomes) are complementary. To get the best appreciation for how these 2 approaches to causal inference work together I would recommend reading Morgan & Winship "Counterfactuals and Causal Inference". From this book you will learn that there are 3 main ways to estimate causal effects: 1) backdoor criterion, 2) instrumental variables, and 3) front-door criterion.

While there are 3 methods, the overwhelming majority of causal inference journal articles in econometrics and social science use method#2: instrumental variables. Incidentally, for the instrumental variable approach the DAG is in many ways unnecessary because it always has the same skeleton and can be easily described in words. One may say that for IV a DAG is crucial; yes it is because it must look like the one below (Z is the instrument). But since all IV DAGs must look like this, how crucial is the DAG in IV and what role does it play other than being a nice visualization? enter image description here A DAG is crucial for method#1: backdoor criterion. But in practice, it is difficult to argue convincingly that a suitable DAG has been constructed. In econometrics or social science you will hardly find a journal article using this method. And if you do, it is almost certainly not going to contain a DAG. From what I've seen this method is used successfully quite often in the medical field. For method#3: front-door criterion the DAG is usually relatively simple, with only a couple front-door paths, and thus it can be easily described in words. So, at the end of the day, the graphical approach is a nice add-on but unless you are estimating causal effects using backdoor criterion (which I find to be rare outside the medical field) or with an elaborate front-door criterion (also relatively rare) a DAG isn't crucial. In contrast, the potential outcomes framework underlies the very substance of causal inference and, frankly, you can't even define causal inference without it. The 2 books by Angrist and Pischke are somewhat unambiguously the best introduction to the potential outcomes approach (in econometrics and social science); the book by Hernan and Robins seems to be highly valued as well, particularly in public health/medical field (but I haven't read it in full). What I consider to be the most valuable contribution of the graphical approach camp is to raise awareness around collider variables; some of its implications (e.g. endogenous selection bias) are critical, top of mind considerations in causal inference across disciplines.

My favorite resources:

(diagram credit: https://donskerclass.github.io/EconometricsII/ControlandIV.html)

Improve this answer
$\endgroup$
11
  • 4
    $\begingroup$ DAGs are absolutely necessary for IV approaches; the exclusion restriction is a causal assumption that can only be verified and properly articulated using graphs, the same as ignorability. Plus conditional exclusion (i.e., exclusion restriction conditional on covariates) also requires the same understanding of confounding that the backdoor criterion uses. Just because econometricians haven't used DAGs with IVs doesn't mean they shouldn't or don't have to. It's also worth remembering that DAGs are a way to visualize structural equations, which are at the true core of causal inference. $\endgroup$
    – Noah
    Commented Mar 18, 2022 at 21:21
  • 1
    $\begingroup$ See what i said about DAGs for IV; yes they are absolutely essential but can also quite simply explained with words. What is the value add of DAGs in IV? Minor. It is a nice visual but that's about it. $\endgroup$
    – ColorStatistics
    Commented Mar 18, 2022 at 21:35
  • 5
    $\begingroup$ Structural equations are explicitly nonparametric. I'm not talking about structural equation modeling. Many things can be explained with words; DAG and structural equations are formalizations of the assumptions essential to the methods. $\endgroup$
    – Noah
    Commented Mar 18, 2022 at 22:53
  • 1
    $\begingroup$ That is, to find a conditional IV, you need to find a set S that blocks potential confounding between the instrument and the outcome, as well as violations of the exclusion restrictions (the instrument affecting the outcome other than through the treatment). And for that, you are back to DAGs, just as in the backdoor case. $\endgroup$
    – Carlos Cinelli
    Commented Mar 19, 2022 at 3:49
  • 2
    $\begingroup$ Thank you, Carlos. I seem to be running exclusively into the "classical" IV DAG but perhaps I am exposed to an unrepresentative sample of studies dominated by natural experiments (e.g. Angrist using Vietnam era draft lottery as instrument). In such cases, I think you'll agree that the DAG is largely unnecessary. I am intrigued by these non-classical IV DAGs. As one of the main experts in causal inference with DAGs, you would know better. Can you point me to a journal article with a very different DAG than the classical case? I'll read it and see how that changes my perspective. $\endgroup$
    – ColorStatistics
    Commented Mar 19, 2022 at 4:01
2
$\begingroup$

Simple answer. Yes - the scientific method is there to help FIND and confirm causal relationships. It requires an hypothesis, then a controlled experiment, then a (statistically verified) conclusion. As data are gathered to support the hypothesis, it becomes more and more certain that "x" causes "y".

Improve this answer
$\endgroup$
2
  • $\begingroup$ I realized this as a potential answer in the process of writing my question, but I decided to post my question anyways because my aim was to find something that studies causation itself as the central notion. You could push back and maybe say physics itself is then what I'm looking for. Again, fair enough. I'll actually upvote this because I consider this a valid answer to my query (although other people can decide for themselves and do as they please). $\endgroup$
    – MaximusIdeal
    Commented Mar 19, 2022 at 22:41
  • 1
    $\begingroup$ Agreed: Mill's Methods, as a summary of the causal aspect of the scientific method, have been around for centuries (literally), as a means of getting at causality via the scientific method. What's exciting about the New Causal Revolution is that it is possible, in some scenarios, to get causal information from mere observational studies: statistics had, before the New Causal Revolution, declared that causality was only available in experiments. $\endgroup$
    – Adrian Keister
    Commented Mar 20, 2022 at 1:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.