22
$\begingroup$

Recently, I have been studying autoencoders. If I understood correctly, an autoencoder is a neural network where the input layer is identical to the output layer. So, the neural network tries to predict the output using the input as golden standard.

What is the usefulness of this model? What are the benefits of trying to reconstruct some output elements, making them as equal as possible to the input elements? Why should one use all this machinery to get to the same starting point?

Improve this question
$\endgroup$
1

4 Answers 4

Reset to default
21
$\begingroup$

Auto encoders have an input layer, hidden layer, and an output layer. The input is forced to be as identical to the output, so its the hidden layer we are interested in.

The hidden layer form a kind of encoding of the input. "The aim of an auto-encoder is to learn a compressed, distributed representation (encoding) for a set of data." If input is a 100 dimensional vector, and you have 60 neurons in the hidden layer, then the auto encoder algorithm will replicate the input as a 100 dimensional vector in the output layer, in the process giving you a 60 dimensional vector that encodes your input.

So the purpose of auto encoders is dimensionality reduction, amongst many others.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thanks @Madhulika, maybe I've understood. The algorithm works as follows: it has an input layer, it trains the neural network in order to have an output layer identical to the input layer. Than it compares the input layer with the output layer, and if they are different, it keeps on re-train the neural network. It stops when they are identical. When it finishes, we take the last hidden layer as the best dimensionality reduction approximation of the input layer, and use it for any goal we need. Is this correct? $\endgroup$
    – larry
    Commented Jan 16, 2014 at 17:10
  • $\begingroup$ Yup, you got that nearly right. :) Read some further literature on it. $\endgroup$
    – Madhulika Mukherjee
    Commented Jan 23, 2014 at 9:35
7
$\begingroup$

It can also model your population so that when you input a new vector, you can check how different is the output from the input. If they're "quite" the same, you can assume the input matches the population. If they're "quite" different, then the input probably doesn't belong to the population you modeled.

I see it as a kind of "regression by neural networks" where you try to have a function describing your data: its output is the same as the input.

Improve this answer
$\endgroup$
5
$\begingroup$

Maybe these pictures give you some intuition. As commenter above said auto encoders tries to extract some high level features from the training examples. You may see how prognostication algorithm is used to train each hidden level separately for the deep NN on the second picture.

enter image description here

enter image description here

Pictures are taken from Russian wikipedia.

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ Comment to the pictures would be helpful. $\endgroup$
    – Tim
    Commented Mar 23, 2019 at 18:33
0
$\begingroup$

In terms of ML, features are gold. Learnt features that use as little data as possible but contain as much information as possible enable us to complete many tasks. Auto encoding is useful in the sense that it allows us to compress the data in an optimal way (that can actual used to represent the input data, as observed by the decoding layer).

Now that we have these features, we are able to complete many different tasks - for example we can use it as a very good starting point for supervised learning tasks.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.