We have already worked in predicting chemical compound activity using "classical" neural networks. Now there is all this hype about deep learning. I wonder if you know cases where predictive ability has dramatically increased going from classical to deep neural networks.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
"Dramatic" is subjective.
One example where deep networks do better is the MNIST hand-written digit set. Yann LeCun keeps a web page of progress made by various techniques with many useful links. This is a relatively simple classification problem compared with speech recognition and image recognition, and deep networks are expected to outperform single hidden layer neural networks (with the same number of parameters) by even more on more complicated tasks.
answered Dec 23, 2012 at 23:21
-
1$\begingroup$ yes this is a good example, although it seems that in this case the convolutional networks perform even better $\endgroup$ Commented Dec 24, 2012 at 8:33
-
$\begingroup$ Yes. Convolutional networks may be more efficient for the number of parameters and they may be easier to train. They may also be viewed as examples of deep neural networks. $\endgroup$ Commented Dec 24, 2012 at 11:05
$\begingroup$
$\endgroup$
In my opinion, some of the most convincing results from the deep learning community in the past few years have come from the area of automatic speech recognition (ASR).
At this point, ASR has seen about four decades of excellent work from a number of really smart people, so the field has been on something of a plateau for the past 10 years or so. For example, it's generally considered publishable in this field if you get a result with one-half percent decrease in word error rate over the state of the art.
However, results using deep models have seen remarkable progress in the field in the past few years. Notably, deep models have resulted in a 5% decrease in word error rate for some systems. In addition, deep models appear to be able to learn appropriate feature representations from simpler encodings of speech ; that is, instead of using a typically hand-coded pipeline transforming speech waveforms into mel-frequency cepstral coefficients (MFCCs), deep models appear to be able to learn effective representations of speech data solely from the data, thus removing the need to hard-code these cepstral (or other) representations. These results are remarkable given the historical progress in the field.
Sample references
L Deng et al. "Recent Advances in Deep Learning for Speech Research at Microsoft." ICASSP 2013.
A-R Mohamed et al. "Deep belief networks using discriminative features for phone recognition." ICASSP 2011.
G Dahl, T Sainath, G Hinton. "Improving Deep Neural Networks for LVCSR using Rectified Linear Units and Dropout." ICASSP 2013.
