Can anyone tell me what is meant by the phrase 'weak learner'? Is it supposed to be a weak hypothesis? I am confused about the relationship between a weak learner and a weak classifier. Are both the same or is there some difference?

In the adaboost algorithm, T=10. What is meant by that? Why do we select T=10?

  • Welcome to the site, @vrushali. I edited this to make the English smoother; please make sure it still says what you mean. Also, I am not sure if the second set of questions (about adaboost) is the same as the first set of questions; it may make more sense to separate them into different threads. – gung Jan 13 '14 at 4:34

A 'weak' learner (classifer, predictor, etc) is just one which performs relatively poorly--its accuracy is above chance, but just barely. There is often, but not always, the added implication that it is computationally simple. Weak learner also suggests that many instances of the algorithm are being pooled (via boosting, bagging, etc) together into to create a "strong" ensemble classifier.

It's mentioned in the original AdaBoost paper by Freund & Schapire:

Perhaps the most surprising of these applications is the derivation of a new application for "boosting", i.e., converting a "weak" PAC learning algorithm that performs just slightly better than random guessing into one with arbitrarily high accuracy. --(Freund & Schapire, 1995)

but I think the phrase is actually older than that--I've seen people cite a term paper(?!) by Michael Kearns from the 1980s.

The classic example of a Weak Learner is a Decision Stump, a one-level decision tree (1R or OneR is another commonly-used weak learner; it's fairly similar). It would be somewhat strange to call a SVM a 'weak learner', even in situations where it performs poorly, but it would be perfectly reasonable to call a single decision stump a weak learner even when it performs surprisingly well by itself.


Adaboost is an iterative algorithm and $T$ typically denotes the number of iterations or "rounds". The algorithm starts by training/testing a weak learner on the data, weighting each example equally. The examples which are misclassified get their weights increased for the next round(s), while those that are correctly classified get their weights decreased.

I'm not sure there's anything magical about $T=10$. In the 1995 paper, $T$ is given as a free parameter (i.e., you set it yourself).

  • As far as I know a DecisionStump is different from 1Rule. A Decision Stump is always a binary 1-level tree (for both nominal and numeric attributes). 1Rule can have more than 2 children (for both nominal and numeric) and for numeric attributes have a more complex test than binary split by a value. Also, in WEKA there are 2 different implementations: DecisionStump and OneR. – rapaio Jan 13 '14 at 10:11
  • Hmmm...I guess you're right. The original 1R paper says "The specific kind of rules examined in this paper, called 1-Rules, are rules that classify an object on the basis of a single attribute (i.e., they are 1-level decision trees." but decision trees can be implemented in a lot of different ways. I'll edit clear that up. – Matt Krause Jan 13 '14 at 17:42
  • There is also a native OneR implementation: The OneR package, on CRAN: CRAN.R-project.org/package=OneR, here is the vignette: cran.r-project.org/web/packages/OneR/vignettes/OneR.html (full disclosure: I am the author of this package). – vonjd Sep 1 '17 at 9:41

Weak learner is a learner that no matter what the distribution over the training data is will always do better than chance, when it tries to label the data. Doing better than chance means we are always going to have an error rate which is less than 1/2.

This means that the learner algorithm is always going to learn something, not always completely accurate i.e., it is weak and poor when it comes to learning the relationships between $X$ (inputs) and $Y$ (target).

But then comes boosting, in which we start by looking over the training data and generate some distributions, then find some set of Weak Learners (classifiers) with low errors, and each learner outputs some hypothesis, $H_x$. This generates some $Y$ (class label), and at the end combines the set of good hypotheses to generate a final hypothesis.

This eventually improves the weak learners and converts them to strong learners.

For more information: https://youtu.be/zUXJb1hdU0k.

  • Welcome to CV. Since you're new here, you may want to take our tour, which has information for new users. . This answer does not seem to provide something new or improve over the previous answers. Do you think there is something missing in the previous ones? – T.E.G. Mar 17 '17 at 0:56
  • why should it be below 1/2. If the error rate is above 1/2 it should be weak classifier, too. – Code Pope Jul 19 at 16:45
  • @CodePope , I got your point, but actually a "weak learner" is formally defined in such terms. I agree any model which has error more than 50% is also poor and weak as well. But speaking of formal definitions as defined by scientists, a weak learner is one which has error less than 1/2 or 50%. – Anish Singh Walia Jul 21 at 7:18

Weak learner is the same as weak classifier, or weak predictor. The idea is that you use a classifier that is, well..., not that good, but at least better than random. The benefit is that the classifier will be robust in overfitting. Of course you don't use just one but a large set of those, each one slightly better than random. The exact way you select/combine them depends on the methodology/algorithm, e.g. AdaBoost.

In practice as weak classifier you use something like a simple threshold on a single feature. If feature is above the threshold then you predict it belongs to the positives otherwise you decide it belongs to the negatives. Not sure about the T=10, since there is no context, but I can assume it is an example on thresholding some feature.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.