6
$\begingroup$

I am interested in finding out the graphical causal structure. Causal Discovery algorithms (e.g., DAG learning) are used to identify potential causal graphs. In score-based causal discovery methods, many model scores such as AIC, BIC, and DIC have been utilized to compare models and to pick up the best "causal" graph. For example, Greedy Search Algorithm tries to find the minimal BIC's graph as a potential causal graph.

But, I am still not sure about how they can use such scores for "causality"-based model comparison. I know that such scores may represent the better "goodness of fit" but I cannot find out some materials that such scores may represent "causal relationships". Could you help me to understand why such scores can imply causal structures?

There are a lot of causal discovery / DAG learning packages to use such ICs to find out causal graphs.

e.g., causal-learn

"The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC) [2]." from a Medium article

$\endgroup$
9
  • 8
    $\begingroup$ They don't. They are purely statistical measures of association. (Which of course does not preclude using them in a screening step in a causal analysis.) That at least is my understanding, maybe someone has found a way to interpret ICs causally... I would be interested in learning something new here. $\endgroup$ Jul 12 at 14:09
  • 3
    $\begingroup$ As @StephanKolassa said. The various IC s are methods for fitting the best model. If it's also causal, then ... they pick good causal models. $\endgroup$
    – Peter Flom
    Jul 12 at 14:25
  • 7
    $\begingroup$ I will be brutally honest here: most Python libraries are written by computer scientists or software engineers, and that is usually a Good Thing. However, no matter how many "Machine Learning" courses they have taken (or published), they are not statisticians, and most do not display a deep understanding of statistics or the difference between statistical association and information criteria. There, I've said it. $\endgroup$ Jul 12 at 15:41
  • 1
    $\begingroup$ I agree with the other comments - no causality is proven. For causality, you really need a causal diagram / model. There's a great book "They Book of Why" which discusses several examples - how using tar in the lungs (if I recall correctly) as a mediator for the effect of smoking on lung cancer disproved the common cause hypothesis. $\endgroup$
    – MikeP
    Jul 12 at 15:49
  • 7
    $\begingroup$ Lots of great comments here. The idea that causality can be learned solely from data is laughable. $\endgroup$ Jul 12 at 16:04

1 Answer 1

9
$\begingroup$

There is nothing inherently causal about any score. A score encodes assumptions about the underlying model. If the assumptions are met, a score can yield a causal model.

Score-based causal discovery typically starts with assumptions about the joint probability distribution of the variables. Given a set of assumptions, one can then devise a score that, ideally, is optimal for the ground-truth DAG, and worse for anything else. If this is the case, the scenario is called "identifiable" by this score. Even then, algorithms might not find a DAG with an optimal score at all, because the search space is usually gigantic if there are more than a just a handful of variables. This motivates algorithms like greedy equivalence search (GES) which are faster at the expense of exploring only a small part of the search space.

Unfortunately, in many cases the underlying DAG is not identifiable by any score even if one were able to traverse the entire search space. For example, only the Markov equivalence class of the DAG may be identifiable with a given score, and all DAGs that are members of the equivalence would receive the same optimal score. The members of the Markov equivalence class include in particular all supergraphs of the actual DAG. This is a problem, because the candidate graphs returned tend to be very dense. To avoid excessively dense graphs, many methods use some form of penalization in addition to a score. Most scores are based on the likelihood, which makes the AIC or BIC a convenient and popular extension that, unlike for example the L1 norm, does not depend on the magnitude of individual coefficients which are scale-sensitive.

In summary, there is nothing inherently "causal" about the information criteria, or indeed any score. A score has to match the data generating process to be useful. In addition, the score may be adapted to favor preferred outcomes such as sparse solutions.

Finally, since I saw some strong opinions expressed in the comments, let me also point to chapters 2-4 of the Elements of Causal Inference for anyone interested in additional context or a more formal treatment of the subject.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.