Outlier Detection Using VAE Latent Space

Question

I am trying to perform outlier detection using VAE. Before I was performing the same task using normal autoencoder and I used reconstruction error, I trained the network, then I passed new samples as input and marked those with high reconstruction error as outliers.

I know that similar things can be done with VAE, but I was wondering if I could somehow use information from LATENT SPACE? An encoder is learning to get means and STDs that are close to a normal distribution and then decoder to sample from those and recreate input.

But I was wondering if now after training I can pass a new sample through just ENCODER, obtain latent space variables, and based on them decide somehow if the sample is an outlier or not?

I thought about sampling from obtained distribution, checking how far from 0 is the point and if far then marking this sample as an outlier, but using sampling feels a bit random. Or maybe most outlying samples should return after encoding most "unnormal" mean and std?

Answer 1

One concern I would have about this approach is that I believe it's been shown that powerful enough decoders may essentially ignore the stochasticity of the latent space, so your encodings may not need to be very good.

That said, let's assume you got a good reconstruction with a weak decoder. I would say this is doable in that case. I would think a good measure would be to just measure the mean Kullback-Leibler Divergence between your candidate outlier and either some representative subset of known good encodings of that class (e.g. training data - but if you can take a peek at decent production embeddings, should be fine too), or to approximate the whole latent cluster's mean and covariance matrix and KLD against that.

It's more or less the same approach as training the VAE in the first place, only you're comparing against the cluster parameters as your ideal, rather than the (0, 1) Standard Normal parameters.

The first approach seems like it could be a bit more robust, but is far more of a pain to actually crunch the numbers for, the latter introduces the risk that your estimate of the cluster parameters is off. Finally, if you're feeling like something really cheap and simple and likely to go off the rails, you could just use a plain Euclidean distance from the mean, but that's really hacky.

Answer 2

That's an interesting approach and I think you can try it. If indeed VAE works as expected, the latent space has to be the multivariate normal distribution, after all the objects are generated by sampling random numbers from this distribution, i.e., after the training, the input to the decoder is random numbers sampled from a normal distribution.

So one approach is to pass your object to the encoder, and check how far this output is from a normal distribution. You can use a t-test for that. see here for more details regarding the t-test for multivariate normal distribution.

Another approach can be to use a similar approach to VAE-VQ. In this approach, the output of the encoder is clustered using a vector quantizer. After having the clusters you can check the distance from these clusters. See more details here.