They all seem to represent random variables by the nodes and (in)dependence via the (possibly directed) edges. I'm esp interested in a bayesian's point-of-view.


A Bayesian network is a type of graphical model. The other "big" type of graphical model is a Markov Random Field (MRF). Graphical models are used for inference, estimation and in general, to model the world.

The term hierarchical model is used to mean many things in different areas.

While neural networks come with "graphs" they generally don't encode dependence information, and the nodes don't represent random variables. NNs are different because they are discriminative. Popular neural networks are used for classification and regression.

Kevin Murphy has an excellent introduction to these topics available here.

  • $\begingroup$ nice link. thx $\endgroup$ – suncoolsu Nov 14 '10 at 6:02
  • 1
    $\begingroup$ Thanks for the answer. Like the original asker, though, I'm also wondering where multilevel/hierarchical regression models fit into this picture. (Hierarchical is as defined here: en.wikipedia.org/wiki/Hierarchical_linear_modeling) $\endgroup$ – Yang Feb 9 '12 at 2:53
  • $\begingroup$ but there are also generative neural network models. RNNs, GANs, etc. $\endgroup$ – Alexander Reshytko Mar 5 '17 at 16:34

As @carlosdc said, a bayesian network is a type of Graphical Model (i.e., a directed acyclic graph (DAG) whose structure defines a set of conditional independence properties). Hierarchical Bayes Models can also be represented as DAGs; Hierarchical Naive Bayes Classifiers for uncertain data, by Bellazzi et al., provides a good introduction to classification with such models. About hierarchical models, I think many articles can be retrieved by googling with appropriate keywords; for example, I found this one:

C. H. Jackson, N. G. Best and S. Richardson. Bayesian graphical models for regression on multiple data sets with different variables. Biostatistics (2008) 10(2): 335-351.

Michael I. Jordan has a nice tutorial on Graphical Models, with various applications based on the factorial Hidden Markov model in bioinformatics or natural language processing. His book, Learning in Graphical Models (MIT Press, 1998), is also worth reading (there's an application of GMs to structural modeling with BUGS code, pp. 575-598)


Neural networks does not require priors, but each hidden node (neurons) of a neural network can be considered as CPD - Noisy OR/AND CPD for a linear node - Sigmoid CPD for a logistic node

So, neural networks could be viewed as multiple layers of hidden nodes, each with linear/sigmoidal CPDs

Koller's class on Coursera OR her textbook should be a good reference for types of CPDs.

  • $\begingroup$ What does CPD stand for? $\endgroup$ – gwr Oct 5 '16 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.