1
$\begingroup$

I'm currently creating an autoencoder that reduces a dataset to single a single dimension. For example: If the input is 5, the autoencoder will have the following topology:

  • Input Layer: 5 dimensions
  • Hidden Layer: 1 dimension
  • Output Layer: 5 dimensions

What would be the intuition behind this approach since I'm not having multiple layers? Also, will there be some effect in the case of really high dimensional data?

$\endgroup$

1 Answer 1

0
$\begingroup$

by minimizing squared reconstruction error, you are basically looking for a latent representation that explains the most variance in the data. If the hidden neuron has a linear activation function, this will be the same as calculating the first principal component of your dataset. If you want to get more intuition into this, look for an explanation of PCA and the relationship between autoencoder and PCA.

If the hidden neuron is not linear, then the result will be something weird, since with one hidden layer and neuron we cannot learn a meaningful non-linear mapping between the hidden layer and the data. If you have multiple hidden layers with non-linear activation functions, then the non-linear function of these components will explain the most variance in the data, in contrast to principal components that are only related to the data linearly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.