When pre-training deep neural networks layer by layer, is it normal to pre-train the layers -which haven't been pre-trained by unsupervised training- by using supervised training before we train the whole network using supervised training? Should we pre-train the classification layers alone, keeping the unsupervisedly trained layers locked, before we unlock them and train the whole network by supervised training?
I can imagine that in the initial stage of supervised learning the features might get altered a lot, due to the fact that the last layer hasn't even started to converge yet when we would perform supervised learning without pre-training the classification layers. This would then radically alter the unsupervisedly learned features based on faulty classification layers which haven't converged yet.
On the other hand, when we would pre-train the classification layers using supervised learning, we might get stuck in a bad local optimum. Because the classification layers have already converged, it is likely that supervised training on the whole network will also have its local optimum nearby.
Is it a good idea to perform pre-training on the classification layers alone, before performing supervised training on the whole network?