I have read and heard in several places that Deep Learning Networks take considerably longer to train than, say, support vector/kernel machines, random forests or boosting methods, but they can give better performance.
My question is, what is fundamentally different about deep learning networks in relation to other learning methods that explains this difference? What is it known about their training complexity?
For example, are DNN discriminative (in contrast to the other methods)? It looks like they could be considered generative since Wikipedia mentions:
Once sufficiently many layers have been learned the deep architecture may be used as a generative model by reproducing the data when sampling down the model (an "ancestral pass") from the top level feature activations.
I have also read on Wikipedia that DNNs can be pre-trained in an unsupervised manner. Is this a notable difference in relation to other methods. Is this pre-training step that common? But most importantly, why do DNNs take so long to train in comparison to other methods?