With all the media talk and hype about deep learning these days, I read some elementary stuff about it. I just found that it is just another machine learning method to learn patterns from data. But my question is: where does and why this method shine? Why all the talk about it right now? I.e. what is the fuss all about?
$\begingroup$
$\endgroup$
1
asked Feb 17, 2014 at 13:40
-
4$\begingroup$ Look at Geoff Hinton's and Andrew Ng's qualitative youtube lectures for an easy overview of why it's so good. $\endgroup$– JaseCommented Feb 17, 2014 at 13:56
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
The main purported benefits:
(1) Don't need to hand engineer features for non-linear learning problems (save time and scalable to the future, since hand engineering is seen by some as a short-term band-aid)
(2) The learnt features are sometimes better than the best hand-engineered features, and can be so complex (computer vision - e.g. face-like features) that it would take way too much human time to engineer.
(3) Can use unlabeled data to pre-train the network. Suppose we have 1000000 unlabeled images and 1000 labeled images. We can now drastically improve a supervised learning algorithm by pre-training on the 1000000 unlabeled images with deep learning. In addition, in some domains we have so much unlabeled data but labeled data is hard to find. An algorithm that can use this unlabeled data to improve classification is valuable.
(4) Empirically, smashed many benchmarks that were only seeing incremental improvements until the introduction of deep learning methods.
(5) Same algorithm works in multiple areas with raw (perhaps with minor pre-processing) inputs.
(6) Keeps improving as more data is fed to the network (assuming stationary distributions etc).
$\begingroup$
$\endgroup$
4
Another important point in addition to the above (I don't have sufficient rep to merely add it as a comment) is that it is a generative model (Deep Belief Nets at least) and thus you can sample from the learned distributions - this can have some major benefits in certain applications where you want to generate synthetic data corresponding to the learned classes/clusters.
answered Sep 22, 2014 at 9:37
-
1$\begingroup$ This is not a general property of deep learning, but of the concrete model in question. E.g. you can sample from a Gaussian but not from a logistic regression. You can also sample from a variety of deep learning models, e.g. Deep belief nets as you said, deep Boltzmann machines, Deep latent Gaussian models, etc. But you cannot sample from drednets, which are the models that have been used in all the impressive applications. $\endgroup$– bayerjCommented Sep 22, 2014 at 10:36
-
$\begingroup$ ... Why can't you sample from a logistic regression? $\endgroup$– Hong OoiCommented Sep 22, 2014 at 10:46
-
$\begingroup$ Given an LR model $p(c|x)$, you can sample the class conditioned on the input. But you cannot sample an input. Thus, it is more of a discriminativ/generative distinction. $\endgroup$– bayerjCommented Sep 22, 2014 at 11:30
-
1$\begingroup$ But the same holds for a gaussian regression model, if by that you mean basic linear regression. $\endgroup$– Hong OoiCommented Sep 22, 2014 at 11:52