Let me first say I (think) understand the procedural and topological differences between an autoencoder and the convolutional layer of a network.

A convolutional layer reduces dimensionality by effectively assuming there is valuable localised structure in the input data. To my eye convolutional network layers seem a little like using a mini neural networks (and splitting the input into mini inputs) and the computational time saving is done by virtue of the mini neural nets not interacting until the pooling layers. The mini nets use back propagation to learn what to look for - what local features are of value.

Auto encoders can reduce dimensionality by having a smaller number of hidden units than input units. They almost seem like the hidden units are a compression algorithm. Auto encoders use back propagation to find the best weights to match the outputs to the inputs. In a sparce auto encoder each hidden unit seems to look for a specific feature of the input and then actives if that feature is present.

So my question is; are convolutional layers a type of auto encoders, are auto encoders a type of convolutional layer or do these things just sound the same but do a very different task?

  • $\begingroup$ the convolution layer is not reducing dimensionality in the same way as the auto-encoder. The AE tries to recreate the input from a smaller hidden layer and tries to minimize the error of that smaller representation. CNNs, do not have this aim $\endgroup$ – Vass Mar 23 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.