Does information criteria (AIC, BIC and DIC...) imply "causality"?

Question

I am interested in finding out the graphical causal structure. Causal Discovery algorithms (e.g., DAG learning) are used to identify potential causal graphs. In score-based causal discovery methods, many model scores such as AIC, BIC, and DIC have been utilized to compare models and to pick up the best "causal" graph. For example, Greedy Search Algorithm tries to find the minimal BIC's graph as a potential causal graph.

But, I am still not sure about how they can use such scores for "causality"-based model comparison. I know that such scores may represent the better "goodness of fit" but I cannot find out some materials that such scores may represent "causal relationships". Could you help me to understand why such scores can imply causal structures?

There are a lot of causal discovery / DAG learning packages to use such ICs to find out causal graphs.

e.g., causal-learn

"The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC) [2]." from a Medium article

Answer 1

There is nothing inherently causal about any score. A score encodes assumptions about the underlying model. If the assumptions are met, a score can yield a causal model.

Score-based causal discovery typically starts with assumptions about the joint probability distribution of the variables. Given a set of assumptions, one can then devise a score that, ideally, is optimal for the ground-truth DAG, and worse for anything else. If this is the case, the scenario is called "identifiable" by this score. Even then, algorithms might not find a DAG with an optimal score at all, because the search space is usually gigantic if there are more than a just a handful of variables. This motivates algorithms like greedy equivalence search (GES) which are faster at the expense of exploring only a small part of the search space.

Unfortunately, in many cases the underlying DAG is not identifiable by any score even if one were able to traverse the entire search space. For example, only the Markov equivalence class of the DAG may be identifiable with a given score, and all DAGs that are members of the equivalence would receive the same optimal score. The members of the Markov equivalence class include in particular all supergraphs of the actual DAG. This is a problem, because the candidate graphs returned tend to be very dense. To avoid excessively dense graphs, many methods use some form of penalization in addition to a score. Most scores are based on the likelihood, which makes the AIC or BIC a convenient and popular extension that, unlike for example the L1 norm, does not depend on the magnitude of individual coefficients which are scale-sensitive.

In summary, there is nothing inherently "causal" about the information criteria, or indeed any score. A score has to match the data generating process to be useful. In addition, the score may be adapted to favor preferred outcomes such as sparse solutions.

Finally, since I saw some strong opinions expressed in the comments, let me also point to chapters 2-4 of the Elements of Causal Inference for anyone interested in additional context or a more formal treatment of the subject.