What are the differences of modeling ability between Variational Auto-encoders (VAEs) and Restricted Boltzmann Machines (RBMs)?

What I am interested in is to know about the unsupervised learning power differences. Where exactly VAEs work better than RBMs and vise versa? Also what RBMs can capture that VAEs can not?

I am already aware of this post and I know the concept of VAEs and RBMs. Also this one is very brief and is not what I am looking for.

Does anybody know about any (theoretical) limitations on the modeling power for one that makes another superior?

up vote 2 down vote accepted

Theoretically RBMs are undirected models with no connections between any observed variables or any latent variables, VAEs are directed models with continuous latent variables.

The Deep Learning book (Chapter 20 Deep Generative Models) provides a very good summary of pros and cons of VAEs and all types of RBMs.

To quote some

The VAE framework is very straightforward to extend to a wide range of model architectures. This is a key advantage over Boltzmann machines, which require extremely careful model design to maintain tractability.


The variational autoencoderis defined for arbitrary computational graphs, which makes it applicable to a wider range of probabilistic model families because there is no need to restrict the choice of models to those with tractable mean field fixed point equations.


One very nice property of the variational autoencoder is that simultaneously training a parametric encoder in combination with the generator network forces the model to learn a predictable coordinate system that the encoder can capture. This makes it an excellent manifold learning algorithm.

  • very nice book and very nice quotes! thanks :) – PickleRick Jan 4 '17 at 8:06
  • @Anoosh you're welcome – dontloo Jan 4 '17 at 12:08

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.