4
$\begingroup$

I have seen the term deep autoencoders in a couple of articles such as Krizhevsky, Alex, and Geoffrey E. Hinton. "Using very deep autoencoders for content-based image retrieval." ESANN. 2011.

What's the difference between autoencoders and deep autoencoders?

$\endgroup$
5
$\begingroup$

Autoencoder is basically a technique to find fundamental features representing the input images. A simple autoencoder will have 1 hidden layer between the input and output, wheras a deep autoencoder will have multiple hidden layers (the number of hidden layer depends on your configuration). Refering the first figure below (deep autoencoder), the demsion of the input image (e.g. size=100x100 pixels) is reduced to 2000,1000,500,30(e.g. size = 10x3) respectively; this part is called encoder. Then, the reduced coder layer is reconstruced back to the original image; this part is called the decoder. Typically, you may use the features produced in any layer of the encoder (i.e. 2000,1000,500, or 30) for building a classifier (e.g. softmax classifier). And this classifier performance is generally better than a classifier using original features without any decoding.

enter image description here

You can imagine each hidden layer represent some form of fundamental features that construct the next layer of features. (See below figure; the first layer learn the colour formation; the second layer learns the edges; the third layer learn different parts of face; the fourth layer learn combination of parts to represent face)

enter image description here

References:

  1. Jones, N. (2014). Computer science: The learning machines. Nature, 505(7482), 146-148. doi:10.1038/505146a

  2. Hinton, G., & Salakhutdinov, R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504--507.

A good tutorial learning deep learning: http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

A github for sample solutions for the UFLDL tutorial: https://github.com/johnny5550822/Ho-UFLDL-tutorial

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.