# Questions tagged [variational]

The tag has no usage guidance.

35 questions
Filter by
Sorted by
Tagged with
1 vote
34 views

### Understanding Variational inference and EM in relation to each other

I have read several answers like here but, somehow I still have a few doubts. I hope to present my understanding and ask a few questions to clear my doubts EM: A maximization maximization algorithm E-...
• 2,543
97 views

### Understanding a beta-variational autoencoder

I'm working on a beta-variational autoencoder using car images from the Vehicle Color Recognition Dataset. At this point, I'm just exploring different architectures and values for beta. (If you're ...
• 1,148
25 views

### Prior estimation in Dynamic (sequence to sequence) Variational Autoencoders (DVAE) with LSTMs

I am trying to implement a sequence-to-sequence variational autoencoder that consists of two parallel sequence encoders. One of the encoders is based on a standard normal prior as in the vanilla vae (...
• 81
139 views

### Confusion with the "lower bound"-term in diffusion models

I am trying to understand the maths of diffusion models following this video explanation on youtube and this blog post. Here is what how I understood it so far: The overall goal is, that we want to ...
216 views

### Should the KL loss term for a VAE be the KL-Loss of a batch's mean mu and log sigma, or is it the mean of the kl loss for each individual input image?

I've been trying to learn about Variational Autoencoders and been looking at the Keras sample implementation (https://github.com/keras-team/keras-io/blob/master/examples/generative/vae.py) I'm ...
• 121
22 views

• 936
1 vote
93 views

### In the β-TCVAE paper, can someone help with the derivation (S3) in Appendix C.1?

Paper: Isolating Sources of Disentanglement in VAEs I follow as far as, $$\mathbb{E}_{q(z)}[log[q(z)] = \mathbb{E}_{q(z, n)}[\ log\ \mathbb{E}_{n'\sim\ p(n)}[q(z|n')]\ ]$$ Subsequently, I don't ...
• 23
695 views

### How does maximizing ELBO in Bayesian neural networks give us the correct posterior predictive distribution?

In Bayesian/variational neural networks one often uses the Evidence Lower BOund (ELBO) as the objective function to optimize with respect to the model parameters. That is if $D=\{y_i,x_i\}_{1\dots n}$ ...
• 121
1 vote
115 views

### Plot Latent-Space of VAE with embedding or just z_mean?

My model is build on this architecture from github: https://github.com/arogers1/VAE_LSTM_Text_Encoding/blob/master/vae_lstm.py The parameters I found the best results with, included a latent_dim of 16....
• 11
259 views

### Does VAE backprop start from the decoder all the way to encoder?

In neural networks that start with input layer, run through hidden layers, and ultimately reach the output layer, we start back-propagation from weights closer to output layer and go backward towards ...
• 421
1 vote
354 views

### How disentaglement in latent space can produce poor variety of instances in VAE..?

I'm reading about $\beta$-VAE which essentially proposes a way to disentangle representations in the latent space. We can subjectively (I guess) identify axes carrying specific sources of variations ...
• 617
453 views

### Why do we need Jensen inequality for variational autoencoders?

Just to clarify, I think I understand all the derivations in context of VAEs pretty well; however, there is one last thing that I need explained. There are multiple related derivations of the evidence ...
• 121
1k views

### VAE loss doesn't converge to zero. Does it make sense to sample new instances from trained latent space?

I aim to use a variational autoencoder (VAE) as a generative model. Does this make sense only if the reconstruction loss converges towards zero? On a project I'm working on, the loss is getting ...
• 617
1 vote
187 views

### Why do we concatenate the condition vector two times in conditional variational autoencoder (CVAE)

I don't quite understand why, in conditional variational autoencoders (CVAEs) we condition on both the encoder and the decoder. In particular, in CVAE the objective function is defined to be: \...
• 617
457 views

### How to choose the number of latent dimensions in VAE?

I have trained a VAE that can generate photos of human faces. I have isolated the dimension that correlates most to smiling and now I only want the VAE to generate smiling faces. May I know is it a ...
• 323
80 views

### VAE divergence is positive in minimization of variational inference?

I have been going through the minimization of Variational inference and have a good understanding of all the steps taken: However, there is a part that relies on KL >= 0: I have derived the ...
1 vote
126 views

### Clarification of Equation for Variational Inference in Pattern Recognition and Machine Learning

I am looking at the derivation of variational inference and specifically the approach taken by Bishop in his book on page 465 as illustrated in the Figure below. The key step is the statement below ...
• 51
111 views

### Rao-Blackwellization in Black Box VI

In the paper, "Black Box Variational Inference," by Ranganath et al. (2013), the authors derive a Rao-Blackwellized estimator of the gradient of the evidence lower bound with respect to a ...
• 41
2k views

### How does the reparameterisation trick work for multivariate Gaussians?

I understand that for sampling from a univariate Gaussian, we can use $x = g(\epsilon) = \mu + \epsilon \sigma$ and then differentiate this transformation with respect to $\mu, \sigma$. How does ...
59 views

• 7,539