I was just reading a paper, seeing someone do the following:
As a pre-processing step they did the following:
PCA the original data -> Stacked Autoencoder
Then they fed this pre-processed data into a feed-forward neural network. But - there are two issues that I have.
Why would they PCA the data? Shouldn't the stacked autoencoder be able to learn the linear representations?
Secondly, I noticed that they used the SAE for pre-processing. I mostly see SAEs for pre-training. That is, for initializing the weights of a feed-forward neural network. Do these "pre-processing" and "pre-training" steps cause different results? What are the implications of "pre-processing" data with the SAE to be fed into a FF instead of "pre-training" the weights of a FF with a SAE?
Thanks in advance.