Autoencoder based anomaly detection: how to train AE also with outliers?

Question

Suppose the data without labels, i.e., unsupervised anomaly detection task. The data are multivariate sequences, so the idea is to use LSTM based autoencoder (AE). However, typically AE-s for anomaly detection are trained on "normal" samples only, meaning that they learn the patterns of the normal data and will not be able to reconstruct anomaly correctly, thus the reconstruction error will be high for outliers. BUT, what if the labels are not known, is it possible to use it anyway? The model is expected to be biased, but still, this is an autoencoder and it should filter out the noise in process of encoding, or? They discuss it in this book Outlier Analysis, but nowhere else.

Plus, how to preprocess the data for such model? Outliers will bias the mean and variance when scaling to zscore..

Answer 1

An auto encoder with one middle layer and linear activation functions all-over, this auto encoder performs a principal component analysis followed by a prediction from the components. Specifically, from the eigenvectors of the covariance matrix associated with these middle-layer nodes.

The confidence intervals of these predictions have been derived. You can use these confidence bounds to demarcate the (multivariate) tails of the distributions associated with your reconstructed outputs. Subsequently, this approach can be used to threshold for outliers.