# Obtaining VAE reconstruction probability

How does one calculate the reconstruction probability? Let's look at the keras example code from here. Is the reconstruction probability the output of a specific layer, or is it to be calculated somehow?

According to the cited paper, the reconstruction probability is the "probability of the data being generated from a given latent variable drawn from the approximate posterior distribution". [2]

[2] Variational Autoencoder based Anomaly Detection using Reconstruction Probability - Jinwon An, Sungzoon Cho

• define reconstruction probability? Commented Sep 17, 2019 at 17:29
• I updated the question. Sorry for the confusion.
– jkf
Commented Sep 23, 2019 at 11:04

Actually, the author of the original paper (Variational Autoencoder based Anomaly Detection using Reconstruction Probability - Jinwon An, Sungzoon Cho) abused the vocabulary. Also note that the author were not consistent when defining the reconstruction probability. In page 8 they defined it as the expectation of the log likelihood under the latent variable. In page 9 they estimated the expectation of the probability.

As a result some papers refer to An and Cho with different understanding of the term reconstruction probability. As an example here is a published work using the definition of page 9 (http://proceedings.mlr.press/v95/guo18a.html) and another paper using the definition of page 8 (https://arxiv.org/pdf/1802.03903.pdf)

Also, note that when you read reconstruction probability you would think that the value is between 0 and 1. But it is not the case. For instance, in case you use a continuous variable for your variational autoencoder output you would use the pdf of that variable. But the density has only the requirement to integrate to 1. Meaning that your "reconstruction probabily" can be very high in certain cases.

At the end, the definition you use depend on your problem. In my case working with distribution over sequences it wasn't stable to use the definition of page 9. Using the log likelihood made more sense. The actual expectation could be approximated using Monte Carlo after having sampled some latent vectors corresponding to the data point of interest.

• (+1) RP is a great idea, but unfortunately it has been ill-defined in that first paper. Commented Mar 24 at 14:24

Look at this tensorflow implementation. https://github.com/Michedev/VAE_anomaly_detection/blob/master/VAE.py

The author 1. build two dense nets to obtain $$\vec \mu$$ and $$\vec \sigma$$ at the last layer of the decoder (line 94); 2. use multivariate_normal.pdf function to compute reconstruction probability (line 161).

• Welcome to Stats.SE. Can you please edit you answer to expand it, in order to include the main points of the link you provide? It will be more helpful both for people searching in this site and in case the link breaks. By the way, take the opportunity to take the Tour, if you haven't done it already. See also some tips on How to Answer, on formatting help and on writing down equations using LaTeX / MathJax. Commented Oct 15, 2019 at 9:03

The reconstruction probability is $$f(x|\vec \mu, \vec \sigma$$) where $$f(\cdot | \vec \mu, \vec\sigma)$$ is the density of a normal distribution with mean $$\vec \mu$$ and diagonal covariance $$\vec \sigma$$.

$$\vec \mu$$ and $$\vec \sigma$$ are indeed the outputs of the encoder part of the VAE. Please refer to an implementation here. The reconstruction probability is calculated using the " reconstructed_probability" function.

• $\vec{\mu}$ and $\vec{\sigma}$ are outputs of the decoder? Not the encoder? Also, is the reconstruction probability a scalar value? If so, how does one obtain it? (I'll need a little guidance, because I have trouble imagining my keras model as a probabilistic model)
– jkf
Commented Sep 25, 2019 at 8:40