What is the objective of a variational autoencoder (VAE)?

Question

I have read a lot of literature on VAE's and I have understood the basic set-up. However, I still don't know what the overall goal is. The basic set-up is that we have a dataset of observations $\pmb{x} = \{x_1, ..., x_n\}$ and a set of latent variables $\pmb{z}$.

My question: What do we want to compute and why? So do we want to compute

• a) The joint probability distribution $p(\pmb{x,z})$ ? and/or
• b) The posterior distribution $p(\pmb{z}|\pmb{x})$?

And what do we use the distributions for? Do we want to generate new data points?

Answer 1

Similar to Auto-encoders, the objective of a Variational Auto-encoder is to reconstruct the input.

The only difference is that AEs have direct links between encoder and decoder parts, but VAEs have a sampling layer which samples form a distribution (usually a Gaussian) and then feeds the generated samples to the decoder part.

Here are some examples from different auto encoders as generative models. You can easily see how the networks are able to capture the data distribution and generate samples very similar to the original ones by only using random observations as an input. On the top, there's the random input and on the bottom, there's the reconstructed image. The models are trained on MNIST.

If you have a look at this paper, you will find the answer to your question:

Answer 2

What do we want to compute and why?

I thought you are mostly interested in the inference issues here. Once we have the trained model we can inference any distribution using the model. For instance:

1. We can do sampling using the joint distribution $P(X,Z)$.

Why? It is a method for use to generate new data points.
How? We first sample from the latent variables $Z$ and then sample from the conditional distribution $P(X|Z)$. If $X$ are all binary varialbes we can just sample according to a binomial distribution.

2. We can plot the conditional expectation $E(X|Z)$.

Why? We can intuit the role of the latent variables by checking the expectation of the manifest variables.
How? We just sample from the latent variables and then get the conditional distribution and that is the expectation of the manifest variables. We then can plot the images on a N-D grid where the grid axes correspond to $z_1$...$z_n$ respectively.

For instance, if we set two latent variables(discretized here) for the MNIST task and we may get such a grid:

We can see that most likel the vertical coyordinate controls the thickness and the horizental coordinate the curve.

3. We can calculate the joint distribution of the manifest varialbes $P(X)$.

Why? We can use this distribution to check the validity of some new data. If the probability of a new case is too low we can say that it an invalid case and otherwise it would most like a similar case.
How? We can use a validation dataset and a test dataset. Firstly over the validation dataset(all valid cases) we calculate the joint probability(of the manifest variables by summing out the latent variables) of each case and then find the 75 percentile and the 25 percentile. Then we try the test cases(may contain invalid cases), if the probability falls in the range(25-75 percentile) we say that it is valid otherwise invalid. In this way we can filter out some bad cases from the test dataset.

Actually once we know the parameters of the Bayes model we can get any distribution and we can use them to find many interesting things.

Answer 3

AVE's are networks that have two parts the encode and decoder. The encoder has a v shape so does the decoder. When they are placed together they have a shape like this ><. The interesting part is the latent vector, which is between the two v's. After you have trained the AVE's (say on faces) the decoder part can be discarded so you are left with the encoder part.

Now the latent vector is the input to the encoder lets say that the latent vector is an array of 100 floating point numbers between -1 and 1. Since you trained you AVE on faces you can generate new faces by creating a random array of floating point numbers between -1 and 1 and generate a new face. This can be used to make all sorts of things like new shoes, bags new maps for games, art, cars. The latent vector is the distribution which is created randomly by you after you have trained the network. The distribution has become the data. With 100 floating point numbers the distribution space is very large that would be if my math is correct 2 to the power 100*31 unique faces. that is a large space you would create faces of people that were never born and those that are dead. Hope that helps you.

Answer 4

The objetive of an autoencoder is to learn an encoding of something (along with its decoding function). There are many uses for an encoding.

In a variational autoencoder what is learnt is the distribution of the encodings instead of the encoding function directly. A consequence of this is that you can sample many times the learnt distribution of an object’s encoding and each time you could get a different encoding of the same object. In this sense, variational autoencoders capture the idea that you can represent something in many ways as long as the “essence” is present in all the encodings. What is “essential” to be represented in each encoding is problem dependent; some problems may require more precision and other problems less. The precision of the encodings can be adjusted by adjusting the neural network used for learning the encoding as well as the cost function used to judge how similar is the reconstruction of the object from the encoding.

Variational autoencoders can learn more complex objects than plain autoencoders given the same amount of data with the trade-off of being less precise (although being less precise is not always a bad thing).