# Removing immoralities in bayesian networks

In many toy examples of bayesian networks, including the ones used in university courses, there are immoralities in the graph (see here, for example). Is it common practice to leave them in the network, provided there are strong motivations to keep them in, or is it best to attempt remove all of them by adding edges?

• Why do you want to remove them? Usually, ones hope is that the qualitative structure of the net reflects the conditional independencies between the vriables Oct 20, 2023 at 10:12
• The idea is that two parents should be somehow connected in a v-structure. But I'm not sure when/if immoralities should be taken care of. Oct 20, 2023 at 10:59
• Some algorithms add edges to remove them for efficient inference, but that is just to aid computations. If there is no direct association between parents I don't see why one should be added - the graph allows users to quickly see independencies and direct associations. Oct 20, 2023 at 11:08
• For others confused as I was, an immorality is when a Bayesian net has two nodes that share a child node, but do not have a "covering" edge between them. Further class slides that define and demonstrate "immoralities". A "Skeleton" is the undirected graph. Related:stats.stackexchange.com/a/270654/147395 Oct 28, 2023 at 20:07

## 1 Answer

A v-structure $$X \rightarrow Z \leftarrow Y$$ is an immorality if there is no direct edge between $$X$$ and $$Y$$. If there is such an edge, it is called a covering edge for the v-structure

- Definition 3.11, Ch. 3 "The Bayesian Network Representation" in [2]

Based on the answer https://stats.stackexchange.com/a/270654/147395 and their referenced book [2] about Probabilistic Graphical Models, including Bayesian networks and structure learning, one cannot learn the directed graph structure from data, and so algorithms will not learn the "covering" edge that connects two unconnected nodes that share a child node. The book they cited references I-equivalence (originally defined in Ch. 3.34 of same book) and then (statistical) identifiability (in bold) in the context of algorithmically learning structure from data:

If our goal is to understand the domain structure, then, clearly, the best answer we can aspire to is recovering $$\mathcal{G}^∗$$. Even here, must be careful. Recall that there can be many perfect maps for a distribution $$\mathcal{P}^∗$$: all of the networks in the same I-equivalence class as $$\mathcal{G}^∗$$. All of these are equally good structures for $$\mathcal{P}^∗$$, and therefore we cannot distinguish between them based only on the data D. In other words, $$\mathcal{G}^∗$$ is not identifiable from the data. Thus, the best we can hope for is an algorithm that, asymptotically, recovers $$\mathcal{G}^∗$$’s equivalence class.

... Therefore, we must generally make a decision about our willingness to include in our learned model edges about which we are less sure. If we include more of these edges, we will often learn a model that contains spurious edges. If we include fewer edges, we may miss dependencies. Both compromises lead to inaccurate structures that do not reveal the correct underlying structure. The decision of whether it is better to have spurious correlations or spurious independencies depends on the application.

- Ch. 18 "Structure Learning in Bayesian Networks" of [2]. Emphasis their own.

With that said, the poster from the related stackexchange question claims it is possible to get experts to inform the graph structure and possibly remove "immoralities" by adding the cover edge. So those toy examples exist as demonstration and would be connected if there were expert knowledge that informed how they should be connected.

To claim expert knowledge or auxiliary knowledge is beneficial to do that which data could not means the expert knowledge has access to information that the model did not, and when that expert knowledge is applied correctly, the model may become identifiable, thus distinguishing between which structure fits the data + expert knowledge and removing possible "immoralities".

So the issue comes down to having enough proper information to resolve the immoralities.

## References

1. Relevant slides open access: https://classes.engr.oregonstate.edu/eecs/winter2020/cs536/slides/undirected3.4pp.pdf
2. Koller, Daphne and Friedman, Nir. "Probabilistic Graphical Models: Principles and Techniques". MIT Press. 2009. ISBN 978-0-262-25835-7 Closed Access URL: https://mitpress.mit.edu/9780262258357/probabilistic-graphical-models/
• By my reading the question was not asking about structure learning. Oct 28, 2023 at 21:44
• I believe they are related. In general, OP want to know whether to leave or remove immoralities in a Bayesian network in practice and the reasoning behind the decision. This addresses that question, which is also of concern to structural learning. The network, presumably, represented some structure based on some reasoning, and so if one is going to change that structure, they need justified reasoning, whether that comes in the form of information from an expert or data. This becomes dependent on what that network represents as edges determine conditional relationships & dependency. Oct 28, 2023 at 22:13