# What is the relation between belief networks and Bayesian networks?

How are Bayesian networks related to deep belief networks? Are they the same?

From the post What is the difference between a neural network and a deep belief network?,
I gathered that deep belief networks are like deep neural networks where the weights and biases are set by iterations of an unsupervised algorithm. (I hope I'm understanding that right anyway.)

## 2 Answers

No, Bayesian network and deep belief network are not the same thing.

• Bayesian network is a type of probabilistic graphical model where vertexes are random variables and edges are conditional dependencies. For large number of random variables, we use the graphical structure assumptions to decompose the joint distribution in a manageable level. In Bayesian network, there are two major tasks, learning and inference. The ultimate goal of learning is getting the joint distribution of the data, and the goal of inferences is trying to calculate the probability for a given events, assuming you already have the model

• As mentioned in the link you provided:

"Neural networks" is a term usually used to refer to feedforward neural networks. Deep Neural Networks are feedforward Neural Networks with many layers.

You can think about neural network is just a big non-linear function, that it can approximate complicated outcomes. Here the nodes are "neurons" and the edges are "connections", where they are essentially building blocks for a function.

• The joint distribution models observe data and unobserved parameters of interest together. We care about the joint distribution because, once we have the joint distribution modeled, we can marginalize the irrelevant data/parameters from the joint distribution and get the distribution of the parameters of interest depending on the problem we are solving.This principle, "modeling the joint distribution and marginalizing the irrelevant to get the desired parameters" is at the heart of Bayesian learning. – Nizam Mar 19 '20 at 5:44

Belief network is a synonym of the Bayesian network, while deep belief networks are a class of deep neural networks(actually hybrid graphical models with both directed edges and undirected edges but are equivalent to the deep neural networks).

Deep neural networks are nothing more than complex functional compositions that can be represented by computation graphs, and the values in their hidden neurons cannot be interpreted as probabilities(often black box), but it can be seen as a process of learning to learn. Belief networks are acyclic directed graphs and the parameters can all be interpreted as probabilities, and they are best for provably correct inference.