18
$\begingroup$

I am looking for papers or texts that compare and discuss (either empirically or theoretically):

with

More specifically, does anybody know of a text that discusses or compares these two blocks of ML methods in terms of speed, accuracy or convergence? Also, I am looking for texts that explain or summarize the differences (e.g. pros and cons) between the models or methods in the second block.

Any pointers or answers addressing such comparisons directly would be greatly appreciated.

$\endgroup$
1
$\begingroup$

Can you be more specific about the types of data you are looking at? This will in part determine what type of algorithm will converge the fastest.

I'm also not sure how to compare methods like boosting and DL, as boosting is really just a collection of methods. What other algorithms are you using with the boosting?

In general, DL techniques can be described as layers of encoder/decoders. Unsupervised pre-training works by first pre-training each layer by encoding the signal, decoding the signal, then measuring the reconstruction error. Tuning can then be used to get better performance (e.g. if you use denoising stacked-autoencoders you can use back-propagation).

One good starting point for DL theory is:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.73.795&rep=rep1&type=pdf

as well as these:

http://portal.acm.org/citation.cfm?id=1756025

(sorry, had to delete last link due to SPAM filtration system)

I didn't include any information on RBMs, but they are closely related (though personally a little more difficult to understand at first).

$\endgroup$
  • $\begingroup$ Thanks @f(x), I'm interested in the classification of (2D or 3D) pixel segments or patches, but I wanted to keep the original question as general as possible. If different methods work best on different types of datasets, I would be interested in a discussion addressing these differences. $\endgroup$ – Amelio Vazquez-Reina Jul 7 '11 at 13:21
  • 1
    $\begingroup$ Generally in terms of image processing, DL methods will act as feature extractors which can then be paired with SVMs to do classification. These methods are generally comparable to hand-engineered approaches like SIFT, SURF, and HOG. DL methods have been extended to video with gated CRBMs, and ISA. Hand-engineered methods include HOG/HOF, HOG3d, and eSURF (see Wang et al. 2009 for a good comparison). $\endgroup$ – user5268 Jul 7 '11 at 15:28
1
$\begingroup$

Great Question! Both adaptive boosting and deep learning can be classified as probabilistic learning networks. The difference is that "deep learning" specifically involves one or more "neural networks", whereas "boosting" is a "meta-learning algorithm" that requires one or more learning networks, called weak learners, which can be "anything" (i.e. neural network, decision tree, etc). The boosting algorithm takes one or more of its weak learner networks to form what's called a "strong learner", which can significantly "boost" the overall learning networks results (i.e. Microsoft's Viola and Jones Face Detector, OpenCV).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.