What's the relation between deep learning and extreme learning machine? Often I have found deep learning and extreme learning machine discussed together.
Based on my little knowledge of the subject my impression is that they are different methods with different aims. 
So what's the relation between deep learning and extreme learning machine ? 
Is there any important paper that I am missing and that I should read?
 A: The difference is: deep learning is original, while ELM is just a fancy name for 3 old methods. 
The “extreme learning machines (ELM)” are indeed worth working on, but they just shouldn’t be called “ELM”. With annotated PDF files at http://elmorigin.wix.com/originofelm , you can easily verify the following facts within 10 to 20 minutes: 


*

*The kernel (or constrained-optimization-based) version of ELM (ELM-Kernel, Huang 2012) is identical to kernel ridge regression (for regression and single-output classification, Saunders ICML 1998, as well as the LS-SVM with zero bias; for multiclass multi-output classification, An CVPR 2007).

*ELM-SLFN (the single-layer feedforward network version of the ELM, Huang IJCNN 2004) is identical to the randomized neural network (RNN, with omission of bias, Schmidt 1992) and another simultaneous work, i.e., the random vector functional link (RVFL, with omission of direct input-output links, Pao 1994).

*ELM-RBF (Huang ICARCV 2004) is identical to the randomized RBF neural network (Broomhead-Lowe 1988, with a performance-degrading  randomization of RBF radii or impact factors).

*In all three cases above, G.-B. Huang got his papers published after excluding a large volume of very closely related literature.

*Hence, all 3 "ELM variants" have absolutely no technical originality, promote unethical research practices among researchers, and steal citations from original inventors. 
A: Extreme learning machines and deep learning are slightly related, but advocate quite adversary concepts.
ELMs are neural nets with a single hidden layer, where the first weight matrix is initialized randomly. This allows the output matrix to be estimated via least squares, which is very quickly done.
Deep learning, on the other hand, is the learning of deep architectures (e.g. deep neural nets). Depending on the strategy, all the layers are optimized jointly or greedily.
Long story short. ELM says: "only learn the last layer". Deep learning says: "Learn all the layers." It seems that DL is much more successfull than ELMs.
