Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm currently creating an autoencoder that reduces a dataset to single a single dimension. For example: If the input is 5, the autoencoder will have the following topology:
What would be the intuition behind this approach since I'm not having multiple layers? Also, will there be some effect in the case of really high dimensional data?
by minimizing squared reconstruction error, you are basically looking for a latent representation that explains the most variance in the data. If the hidden neuron has a linear activation function, this will be the same as calculating the first principal component of your dataset. If you want to get more intuition into this, look for an explanation of PCA and the relationship between autoencoder and PCA.
If the hidden neuron is not linear, then the result will be something weird, since with one hidden layer and neuron we cannot learn a meaningful non-linear mapping between the hidden layer and the data. If you have multiple hidden layers with non-linear activation functions, then the non-linear function of these components will explain the most variance in the data, in contrast to principal components that are only related to the data linearly.
