Are Deep Neural Nets Graphical Models?

In the talk, here at NIPS, they say that:

GANs and VAEs are Graphical Models, just with a particular CPD and cost function. They are bipartite complete graphs.

How can that be explained? I thought that you need probabilities enmeshed in the models, with variables having dependence relationships. In neural nets, they have all sorts of other things like ReLu nodes etc. i.e there are no probability relationships just a series of non linearities alongwith with priors such as regularization or convnet structure.

This seems to be very different view than that is explained at What's the relation between hierarchical models, neural networks, graphical models, bayesian networks? .


You can view a deep neural network as a graphical model, but here, the CPDs are not probabilistic but are deterministic. Consider for example that the input to a neuron is $\vec{x}$ and the output of the neuron is y. In the CPD for this neuron we have, $p(\vec{x},y)=1$, and $p(\vec{x},\hat{y})=0$ for $\hat{y}\neq y$. Refer to the section 10.2.3 of Deep Learning Book for more details.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.