Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
How are Bayesian networks related to deep belief networks? Are they the same?
From the post What is the difference between a neural network and a deep belief network?,
I gathered that deep belief networks are like deep neural networks where the weights and biases are set by iterations of an unsupervised algorithm. (I hope I'm understanding that right anyway.)
No, Bayesian network and deep belief network are not the same thing.
Bayesian network is a type of probabilistic graphical model where vertexes are random variables and edges are conditional dependencies. For large number of random variables, we use the graphical structure assumptions to decompose the joint distribution in a manageable level. In Bayesian network, there are two major tasks, learning and inference. The ultimate goal of learning is getting the joint distribution of the data, and the goal of inferences is trying to calculate the probability for a given events, assuming you already have the model
As mentioned in the link you provided:
"Neural networks" is a term usually used to refer to feedforward neural networks. Deep Neural Networks are feedforward Neural Networks with many layers.
You can think about neural network is just a big non-linear function, that it can approximate complicated outcomes. Here the nodes are "neurons" and the edges are "connections", where they are essentially building blocks for a function.
Belief network is a synonym of the Bayesian network, while deep belief networks are a class of deep neural networks(actually hybrid graphical models with both directed edges and undirected edges but are equivalent to the deep neural networks).
Deep neural networks are nothing more than complex functional compositions that can be represented by computation graphs, and the values in their hidden neurons cannot be interpreted as probabilities(often black box), but it can be seen as a process of learning to learn. Belief networks are acyclic directed graphs and the parameters can all be interpreted as probabilities, and they are best for provably correct inference.
