All Questions
12 questions
0
votes
0
answers
16
views
Why do VAEs work?
I am currently reading into Variational Autoencoders, and although I kind of understand the mathematical background described in the original paper (Auto-encoding Variational Bayes), I am struggling ...
1
vote
0
answers
389
views
Classical VAE not learning 2D gaussian mixture distribution using MSE loss
I've been exploring VAE for non-image data. I consider small to medium-sized continuous vector spaces and I want to learn the distribution of a dataset in that space.
As a warm up exercise, I tried ...
- 113
1
vote
0
answers
148
views
Probabilistic Difference between Autoencoders and Variational Autoencoders
I have recently read up about Autoencoders and Variational Autoencoders. In Variational Autoencoders, the loss is modeled based on what distribution we choose for P(x|z). So, if we choose it to be ...
- 11
3
votes
3
answers
2k
views
In VAE, why use MSE loss between input x and decoded sample x' from latent distribution?
Variational Autoencoders (VAEs) are based on the concept of Variational Inference (VI) and use two Neural Networks similar to Vanilla Autoencoders (AEs) for function approximation. I understood the ...
- 61
1
vote
1
answer
2k
views
Understanding reparameterization trick and training process in variational autoencoders
I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation ...
- 1,030
1
vote
0
answers
66
views
One shot inference with Variational Autoencoders using proposal mean
Let's say you have an already trained Variational Autoencoder where the parameters are $\phi, \theta$ for the recognition and generative models respectively. Let's also assume you have the following ...
- 111
1
vote
1
answer
469
views
Does variational auto-encoder output the variational distribution of the latent variable or the distribution of the input x?
In the simple case of mixture of gaussians(with known variance), we have 2 latent variables $\mu$ and $z$. In the vaiational auto-encoder, we assume that the model is infinite mixture of gaussians. If ...
- 1,382
2
votes
1
answer
2k
views
On evaluating variational autoencoders with prior likelihood and reconstruction error
A common evaluation metric for variational autoencoders (VAEs) is estimating the marginal likelihood of some held-out data, i.e. $p(x)$. This is difficult and often one can only get a lower bound. It'...
- 2,284
14
votes
3
answers
3k
views
Is the optimization of the Gaussian VAE well-posed?
In a Variational Autoencoder (VAE), given some data $x$ and latent variables $t$ with prior distribution $p(t) = \mathcal{N}(t \mid 0, I)$, the encoder aims to learn a distribution $q_{\phi}(t)$ that ...
- 707
1
vote
1
answer
386
views
Discrete Random Variables and Deep Generative Models - Why Gumbel-Softmax is needed?
I am reading this 2014 NIPS paper on deep generative models and their application to latent discrete random variables, and this 2017 ICLR paper on Gumbel-Softmax. I essentially don't understand why we ...
- 1,502
3
votes
0
answers
99
views
Generative Adversarial Network and Variational Autoencoders for Independent Component Analysis?
Background:
I'm working on a model for independent component analysis (ICA) that is based on a methodology similar to GANs and VAEs. What I'm having trouble understanding is how the choice of the loss ...
- 1,368
22
votes
1
answer
6k
views
Why in Variational Auto Encoder (Gaussian variational family) we model $\log\sigma^2$ and not $\sigma^2$ (or $\sigma$) itself?
In theory the encoder in VAE (assuming that variational family is Gaussian) generates the $\mu$ and $\sigma$ (or $\sigma^2$). But, in practice, I have seen people assuming the output is $\log\sigma^2$....
- 1,502