OK, I know there is a lot of topic regarding this in the internet, and trust me, I've googled it. But things are getting more and more confused for me.

From my understanding, Deep Learning (DL) is kind of a subset of Machine Learning (ML) where ML can consist of something like Support Vector Machines and DL can be consists of something like Convolutional Neural Network. Is this correct?

If I want to getting start at this, what should I read first? What kind of research paper that can I read?

  • 2
    $\begingroup$ A very dumbed down explanation would be: A neural network with multiple layers comes under the realm of deep learning $\endgroup$ – Dawny33 Apr 6 '16 at 3:02
  • 8
    $\begingroup$ It is same as asking what is difference between sports and soccer. $\endgroup$ – Vladislavs Dovgalecs Apr 6 '16 at 3:16

Starting with the first page of Goolge Scholar, one finds some promising abstracts.

I. Arel,D. C. Rose, T. P. Karnowski Deep Machine Learning - A New Frontier in Artificial Intelligence Research

This article provides an overview of the mainstream deep learning approaches and research directions proposed over the past decade. It is important to emphasize that each approach has strengths and "weaknesses, depending on the application and context in "which it is being used. Thus, this article presents a summary on the current state of the deep machine learning field and some perspective into how it may evolve. Convolutional Neural Networks (CNNs) and Deep Belief Networks (DBNs) (and their respective variations) are focused on primarily because they are well established in the deep learning field and show great promise for future work.

Yann LeCun, Yoshua Bengio & Geoffrey Hinton, Deep Learning, Nature

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.

@frankov suggested adding this diagram which summarizes one interpretation of the different flavors of machine-learning.

enter image description here


What is Machine Learning?

Machine Learning is a way in which scientists build algorithms that can learn.

  • Choose any Task T, such as driving a car.
  • Then your algorithm can display a certain level of performance P.
  • If this improves with experience E, then the algorithm is said to have...
  • learned!

So what is Deep Learning?

Well, its a bit more complex, but basically let's say your trying to approximate a function. You have inputs and you have outputs. But you don't know what goes on in the middle. For instance, we know humans can drive cars well, they don't crash (often) but we don't really know how the mind is working to achieve this...

So let's think about a neural network model that attempts to drive a car. It is a bit like the brain. So there are many layers to the model. In fact, its a function within a function within a function etc. Because there are many functions nested within one another... we use the term deep to describe it!

In one line:

Deep learning is a kind of neural network, and a neural network is a kind of machine learner. That's it :-)

I recommend this textbook.


Google "machine learning" and you will find a lot of definitions. What is deep learning might be just slightly harder to put a finger on. Everyone will agree that neural nets is a method that is part of machine learning. However 'traditional' neural nets tended to work very poorly ('overtrain') on nets with a large number of hidden layers. Nets with a large number of hidden layers have been (up to now anyway) necessary for solving complicated problems such as image recognition. Deep learning is the name given to a number of techniques for finding the optimal constants in the hidden layers of a neural net so that overtraining doesn't occur in nets with a large number of hidden layers.

  • 1
    $\begingroup$ Telling OP to use Google when OP has already sated that doing so has been unsatisfactory is not really all that helpful. $\endgroup$ – Sycorax says Reinstate Monica Apr 6 '16 at 14:15
  • 1
    $\begingroup$ @C11H17N2O2SNa One of us didn't understand the question. If his question was 'what is ML' then my comment was lame. However, I think the gyst of his question was I understand ML, but what is DL. In which case my comment about googling is just agreeing with the OP. If you will, my answer remains unchanged if the first (objectionable to you) sentence is removed. $\endgroup$ – meh Apr 6 '16 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.