Removing immoralities in bayesian networks

Question

In many toy examples of bayesian networks, including the ones used in university courses, there are immoralities in the graph (see here, for example). Is it common practice to leave them in the network, provided there are strong motivations to keep them in, or is it best to attempt remove all of them by adding edges?

Answer 1

A v-structure $X \rightarrow Z \leftarrow Y$ is an immorality if there is no direct edge between $X$ and $Y$. If there is such an edge, it is called a covering edge for the v-structure

- Definition 3.11, Ch. 3 "The Bayesian Network Representation" in [2]

Based on the answer https://stats.stackexchange.com/a/270654/147395 and their referenced book [2] about Probabilistic Graphical Models, including Bayesian networks and structure learning, one cannot learn the directed graph structure from data, and so algorithms will not learn the "covering" edge that connects two unconnected nodes that share a child node. The book they cited references I-equivalence (originally defined in Ch. 3.34 of same book) and then (statistical) identifiability (in bold) in the context of algorithmically learning structure from data:

If our goal is to understand the domain structure, then, clearly, the best answer we can aspire to is recovering $\mathcal{G}^∗$. Even here, must be careful. Recall that there can be many perfect maps for a distribution $\mathcal{P}^∗$: all of the networks in the same I-equivalence class as $\mathcal{G}^∗$. All of these are equally good structures for $\mathcal{P}^∗$, and therefore we cannot distinguish between them based only on the data D. In other words, $\mathcal{G}^∗$ is not identifiable from the data. Thus, the best we can hope for is an algorithm that, asymptotically, recovers $\mathcal{G}^∗$’s equivalence class.

... Therefore, we must generally make a decision about our willingness to include in our learned model edges about which we are less sure. If we include more of these edges, we will often learn a model that contains spurious edges. If we include fewer edges, we may miss dependencies. Both compromises lead to inaccurate structures that do not reveal the correct underlying structure. The decision of whether it is better to have spurious correlations or spurious independencies depends on the application.

- Ch. 18 "Structure Learning in Bayesian Networks" of [2]. Emphasis their own.

With that said, the poster from the related stackexchange question claims it is possible to get experts to inform the graph structure and possibly remove "immoralities" by adding the cover edge. So those toy examples exist as demonstration and would be connected if there were expert knowledge that informed how they should be connected.

To claim expert knowledge or auxiliary knowledge is beneficial to do that which data could not means the expert knowledge has access to information that the model did not, and when that expert knowledge is applied correctly, the model may become identifiable, thus distinguishing between which structure fits the data + expert knowledge and removing possible "immoralities".

So the issue comes down to having enough proper information to resolve the immoralities.

References

1. Relevant slides open access: https://classes.engr.oregonstate.edu/eecs/winter2020/cs536/slides/undirected3.4pp.pdf
2. Koller, Daphne and Friedman, Nir. "Probabilistic Graphical Models: Principles and Techniques". MIT Press. 2009. ISBN 978-0-262-25835-7 Closed Access URL: https://mitpress.mit.edu/9780262258357/probabilistic-graphical-models/