# Questions tagged [variational-inference]

The tag has no usage guidance.

59 questions
Filter by
Sorted by
Tagged with
37 views

### Posterior approximation following optimization methods

I'm trying to quantify the uncertainty in a high dimensional, and multimodal posterior space. We do not have a analytical solution for the forward model, and the forward model could be expensive to ...
• 21
28 views

• 1,316
59 views

### Understanding Variational inference for LDA

I am trying to derive from scratch variational inference for LDA. I am following this course: https://home.cs.colorado.edu/~jbg/teaching/CSCI_5622/19a.pdf When computing $p(Z|\Theta)$ they do the ...
• 449
441 views

### What is the closed-form of the KL-Divergence between two relaxed Bernoulli distributions?

I've seen in multiple papers that use a relaxation of the Bernoulli distribution as defined in Maddison et. al (here it is referred to as Binary Concrete) and they say that a closed form solution for ...
1 vote
172 views

### Comparing Gibbs sampler and variational inference

I am learning about variational inference and Gibbs simpler. I am in the process of deriving variational inference on my own. In this process, I need to make a comparison with Gibbs sampler. I am ...
• 449
3k views

### What's the role of the commitment loss in VQ-VAE?

I'm reading about VQ-VAE, and trying to understand the commitment loss $\beta||z_e(x) - sg(e)||^2$, described in the following sentence: Finally, since the volume of the embedding space is ...
• 3,340
108 views

### What's the difference between p(Z, X=x) and p(Z|X=x)?

I'm trying to understand variational inference, and I've found resources that mention $p(Z, X=x)$, where $Z$ is a latent random variable and $X$ is the observed random variable. (Here is one such ...
• 221
947 views

I'm reading two descriptions of the VQ-VAE objective: Kingma claims in page 18 that we want to maximize the ELBO, and shows that it can be written as ELBO = logp_{\theta}(x) - KL(q_{\phi}(z|x)||p_{\... • 3,340 2 votes 0 answers 191 views ### why is VQ-VAE considered a variational encoder? I'm reading about VQ-VAE and I'm not sure why do they say we can view it as a VAE. Can you explicitly show: what is the latent z-space - is it the discrete space where z can take the integers 1..K ... • 3,340 1 vote 1 answer 1k views ### VQ-VAE - why do we need to separate the codebook alignment loss and the commitment loss? [duplicate] In VQ-VAE, we separate the codebook alignment loss||sg(z_e(x))-e||^2$and the commitment loss$||z_e(x)-sg(e)||^2$where sg stands for the stop-gradient operator, and the loss is$||sg(z_e(x))-e||^2 ...
• 3,340
120 views

I realized in variational inference, our goal is to approximate $p(z|x)$ with $q(z)$. So we minimize $KL(q(z) || p(z|x)) = \mathbb{E}_{z \sim q} log\frac{q(z)}{p(z|x)}$. We then manipulate, through ...
• 221
1 vote
128 views

141 views

### Difference between KLdiv(P||Q) and KLdiv(Q||P) in variational inference

Variational inference is about finding an estimation Q(z) for the posterior P(Z|x). According to all the variational inference papers, this is done by minimizing the KLdiv(Q||P). I want to understand ...
• 449
1 vote
41 views

### Role of auxiliary objective in semi-supervised VAEs?

In these two papers, mainly: Klys, Jack, Jake Snell, and Richard Zemel. "Learning latent subspaces in variational autoencoders." Advances in neural information processing systems 31 (2018). ...
5k views

### Understanding the Evidence Lower Bound (ELBO)

I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed ...
• 1,787
1 vote
3k views

### VAE: what activation function (if any) to use for the last layer of my decoder if I don't want to assume any knowledge about the scale of my inputs?

I'm working on an implementation of a Variational Autoencoder (VAE). There are lots of helpful examples and guides out there, which typically introduce VAE in the context of image data, e.g. MNIST. ...
• 133
1 vote
35 views

### Why do we use the same parameters for the joint, marginal and conditional distributions in VAEs?

I've noticed in several resources on variational autoencoders (for example the Wikipedia article), we use the same parameters theta ($\theta$) for the prior, likelihood, posterior, etc distributions. ...
• 11
542 views

### Why Reparameterization Trick does not work with discrete latent variables?

I came to know from the Youtube Video here (Timestamp 1:03:55) that Reparameterization trick only works for continuous latent variable. But, I am not clear as to why it does not work for discrete ...
• 421
3k views

### Prior in variational autoencoders

I am currently dealing with variational autoencoders where I've read the original paper "An introduction to variational Bayes" from Kingma and Welling. I am currently still a little confused ...
• 41
1k views

• 617
1 vote
296 views

### Why is KL divergence used as a measure of closeness in variational inference?

I am curious why KL divergence is the standard measure of (dis)similarity used in VI while it is not even a proper metric (asymmetric and does not satisfy triangle inequality).
• 373
1 vote
46 views

### Distribution over parameters vs. distribution over functions

I find it hard to distinguish between these two concepts. In a variational inference setting we learn a distribution over the parameters of our function. in the definition of Gaussian processes we ...
1 vote