# Questions tagged [variational-bayes]

Variational Bayesian methods approximate intractable integrals found in Bayesian inference and machine learning. Primarily, these methods serve one of two purposes: Approximating the posterior distribution, or bounding the marginal likelihood of observed data.

237 questions
Filter by
Sorted by
Tagged with
33k views

### How does the reparameterization trick for VAEs work and why is it important?

How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'?
6k views

### Variational inference versus MCMC: when to choose one over the other?

I think I get the general idea of both VI and MCMC including the various flavors of MCMC like Gibbs sampling, Metropolis Hastings etc. This paper provides a wonderful exposition of both methods. I ...
9k views

### how to weight KLD loss vs reconstruction loss in variational auto-encoder

in nearly all code examples I've seen of a VAE, the loss functions are defined as follows (this is tensorflow code, but I've seen similar for theano, torch etc. It's also for a convnet, but that's ...
6k views

### Relation between variational Bayes and EM

I read somewhere that Variational Bayes method is a generalization of the EM algorithm. Indeed, the iterative parts of the algorithms are very similar. In order to test whether the EM algorithm is a ...
4k views

### What are variational autoencoders and to what learning tasks are they used?

As per this and this answer, autoencoders seem to be a technique that uses neural networks for dimension reduction. I would like to additionally know what is a variational autoencoder (its main ...
7k views

### When should I use a variational autoencoder as opposed to an autoencoder?

I understand the basic structure of variational autoencoder and normal (deterministic) autoencoder and the math behind them, but when and why would I prefer one type of autoencoder to the other? All I ...
2k views

### what does one mean by numerical integration is too expensive?

I am reading about Bayesian inference and I came across the phrase "numerical integration of the marginal likelihood is too expensive" I do not have a background in mathematics and I was wondering ...
8k views

### What is the “capacity” of a machine learning model?

I'm studying this Tutorial on Variational Autoencoders by Carl Doersch. In the second page it states: One of the most popular such frameworks is the Variational Autoencoder [1, 3], the subject of ...
432 views

218 views

### Variational Bayes combined with Monte Carlo

I'm reading up on variational Bayes, and as I understand it, it comes down to the idea that you approximate $p(z\mid x)$ (where $z$ are the latent variables of your model and $x$ the observed data) ...
2k views

### Applying stochastic variational inference to Bayesian Mixture of Gaussian

I am trying to implement Gaussian Mixture model with stochastic variational inference, following this paper. This is the pgm of Gaussian Mixture. According to the paper, the full algorithm of ...
4k views

### What does a 'tractable' distribution mean?

For example, in generative adversarial network, we often hear that inference is easy because the conditional distribution of x given latent variable z is 'tractable'. Also, I read somewhere that ...
737 views

### Variational inference engines

After doing some research on the topic, I have noticed a surprising deficit of inference packages and libraries that rely on message-passing or optimization methods for Python and R. To the best of ...
620 views

### Comparing Laplace Approximation and Variational Inference

Does anyone know of any references that look at the relationship between the Laplace approximation and variational inference (with normal approximating distributions)? Namely I'm looking for something ...
654 views

### choosing prior parameters for variational mixture of Gaussians

I am implementing a vanilla variational mixture of multivariate Gaussians, as per Chapter 10 of Pattern Recognition and Machine Learning (Bishop, 2007). The Bayesian approach requires to specify (...
196 views

### Rao-Blackwellization in variational inference

The Black box VI paper introduces Rao-Blackwellization as a method to reduce the variance of the gradient estimator using score function, in section 3.1. However I don't quite get the basic idea ...
2k views

### Deriving the KL divergence loss for VAEs

In a VAE, the encoder learns to output two vectors: $$\mathbf{\mu} \in\ \mathbb{R}^{z}$$ $$\mathbf{\sigma} \in\ \mathbb{R}^{z}$$ which are the mean and variances for the latent vector $\mathbf{z}$, ...
2k views

### variational inference with KL

i am self-studying variational inference - and in Murphy's book "A probabilistic perspective on machine learning" it is discussed that minimizing the forward KL divergence (which is stated to be zero-...
134 views

### $\min D_\textrm{KL}(p(x_1,\dots,x_n) \mid\mid q_1(x_1)\cdots q_n(x_n))$ gives the marginals of $p(x_1,\dots,x_n)$?

Prove or disprove: Let $p(x_1,\dots,x_n)$ be a given probability distribution over the $n$ variables $x_1, \dots,x_n$. The univariate probability distributions $q(x_1),\dots,q(x_n)$ that minimize the ...
145 views

### Is this problem Bayesian? And can I use variational approximation?

Suppose there are $N$ samples of observations $\mathbf X(n)$ ($n=1,\cdots,N$), which are given by probability distribution $p(\mathbf X(n)|\mathbf Z(n))$ with their conditions are given by hidden ...
574 views

234 views

### What is the idea behind Bayes By Backprop?

Having looked through the internet and the paper, I find Bayes by Backprop very unaccesible for my intermediate understanding of variational inference. Most online guides also lack some explaining ...