In a VAE with Gaussian output the loss function is usually:$$\sum{(\hat x - x)^2} + KL,$$ so the sum of squared errors plus KL divergence. When I also want to predict the variance of the reconstructed input I need 2 outputs for each dimension of x: mean and variance. But in this case how should I calculate the reconstruction error? Should I sample from a normal distribution with the mean and variance I have as current output and calculate sum of squared errors for that sample?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
If you're predicting a mean and a variance with the decoder, you can use a log likelihood loss for the reconstruction loss. So if you predict $\mu_x$ and $\sigma_x$, then the reconstruction loss is:
$$2(\mu_x - x) - 2\sigma_x^2 + \operatorname{log}2\sigma_x + C$$
-
$\begingroup$ Thank you! Just to be sure, so in TensorFlow it would be:
2 * (mu - input) - 2 * tf.square(sigma) + tf.log(2 * sigma)whereinput,muandsigmaare rank 2 tensors with shape (number of samples, input dimensions)? $\endgroup$ Commented Apr 9, 2018 at 7:35 -