Can latent variables be used in Bayesian networks?

Question

I have successfully implemented a hill climbing approach to Bayesian structure learning using a Gaussian Bayesian network. I want to now implement a more sophisticated model with latent variables. However, information on this topic seems very scarce. If I google Bayesian networks, there is an abundance of resources, but almost none for applications of latent variables to BNs.

Can latent variables be used in Bayesian networks? If so, is there an accepted introductory textbook or set of lectures notes that people commonly consult to learn about this? I'm confused why this is hard to find information on. Is it uncommon to do use latent variables with Bayesian networks? If so, why? Are there better alternative methods?

So far, I've been able to find information on hidden Markov random fields, such as

• https://www.fmrib.ox.ac.uk/datasets/techrep/tr00yz1/tr00yz1/node5.html
• https://en.wikipedia.org/wiki/Hidden_Markov_random_field

This is the kind of thing I'm looking for, but I want it specifically for Bayesian networks.

The closest I've found for hidden Bayesian networks has been

Expectation maximization on Bayesian networks with latent variables

Estimating Continuous latent variables in a general Bayesian network

http://maxchickering.com/publications/ml97.pdf

Answer 1

This can absolutely be done and is incredibly widely used. See e.g.

• https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation,
• https://en.wikipedia.org/wiki/Hidden_Markov_model,
• https://en.wikipedia.org/wiki/Kalman_filter
• https://en.wikipedia.org/wiki/Mixture_model.

for a few ubiquitous examples. "Unmeasured confounding" and "latent mediators" are other topics in this area; there are many more. Essentially, any latent-variable problem can be formulated in terms of inference in a Bayesian network.

Kevin Murphy's Probabilistic Machine Learning: Advanced Topics has some sections on this; probably Koller's & Friedman's Probabilistic Graphical Models does as well, but I currently don't have access to that and cannot verify.

Generally speaking, models with latent variables are, of course, significantly more challenging to identify compared to models in which all nodes are observed.