This question stems from Pedro Domingos' excellent paper "A Few Useful Things to Know About Machine Learning." The paper is extremely clear and well-written, but I still have a clarification question. Namely, what is the difference between what Domingos describes on page 1 as a "learner" and a "classifier"? I have largely taken these to by synonymous. The sentence that threw me for a loop in the paper is (again, on page 1): "The test of the learner is whether this classifier produces the correct output yt for future examples xt." I thought this was simply the test of the classifier. Any clarification or further reading suggestions would be greatly appreciated.


I use to find this kind of difference mostly in the world of programmers interested in machine learning. This difference does not appear in the world of statisticians and researchers.

The idea is that a classifier is a program built by a learner. An illuminating intuition comes from one of the many definitions of machine learning which is programming with data. So, you have data (training set) and from that data using a computer program you build another program (for example a decision tree). The program which builds the decision tree from data is the learner. The decision tree is a classifier, because a classifier is a program which is able to predict, which takes only the input data and for each instance it produces the output data.

An alternative way to understand this is that a learner takes the input $x_1,x_2,..,x_p,y$ and produces a classifier. A classifier takes as input $x'_1,x'_2,..,x'_p$ and produces $y'$.

As I said, in research papers this distinction is hard to find. It seems that the researchers are interested only in how to describe the model. When they come to describe how to build that model, then they talk about a learner, and when they talk about how to predict with that model, then they talk about a classifier. So, a third alternative is a functional one. The function of fitting a model is the function of a learner, while the function of predicting values is a function of a classifier.

Note that a regressor is the same as a classifier, only the nature of the output is different.

  • $\begingroup$ Thanks @rapaio. Your functional description near the end finally made things click. Greatly appreciated. $\endgroup$ – kylerthecreator Jul 3 '14 at 11:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.