3
$\begingroup$

I was looking at the denoising autoencoder architecture. It seems that the way it works is that it takes corrupted input and tries to restore it using the error between the uncorrupted and corrupted input. However, how much noise is optimal for an autoencoder? Because if there is too much noise then the autoencoder is just gonna be learning random stuff and if there is too little, it becomes useless.

Thank you

$\endgroup$
4
$\begingroup$

the following paper

P. Vincent, H. Larochelle Y. Bengio and P.A. Manzagol, Extracting and Composing Robust Features with Denoising Autoencoders, Proceedings of the Twenty-fifth International Conference on Machine Learning (ICML‘08), pages 1096 - 1103, ACM, 2008.

contains a variety of images on the MNIST dataset that show how well features are recognized when different levels of noise are added. Especially important are the last pictures where he shows that the more noise is added the better the network learns dependencies between variables. With low noise levels features do not stand out.

The link to the paper is the following: http://www.iro.umontreal.ca/~lisa/publications2/index.php/attachments/single/176

Also note that the noise that is added is not really white noise or anything similar. It is simply setting 20,30,50% of the values at random to zero.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.