I want to train both a single-layer autoencoder and a multi-layer autoencdoer in Keras to reconstruct an input with 24 features, all in the same scale with int values from 0 to ~200000. My question is: what would be the best choice for activation function for each layer for both autoencoders?
In the Keras autoencoder blog post, Relu is used for the hidden layer and sigmoid for the output layer. But using Relu on my input would be the same as using a linear function, which would just approximate PCA. So what would be a better choice to learn non linear features?
Same goes for the multi-layer autoencoder. In the blog post only Relu is used for the hidden layers and sigmoid for the output layer. Do better options for my training data exist?
I plan to use MSE as loss function. Also, Is it necessary to scale/normalize my input data given that all features are already in the same range? And in that case, which scaling would be best?