1
$\begingroup$

I have some questions regarding generalization and recall:

  1. It usually hear that a good classifier should "generalize" from the training data. Is that true?
  2. I feel(?) that a "general" classifier should improve recall, however a "specific" classifier should have more precision, is this true?
  3. If we think of a classifier that has to be learnt from a domain D1 and then used over a different domain D2, then a "generalized" classifier is preferred. In which other situations a "generalized" classifier is preferred?

Thanks in advance and sorry for the vague questions...

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Out of the three points you made, only (1) makes sense. A good classifier that generalizes well, will have high recall and high precision. And there is no such thing as a generalized classifier. $\endgroup$
    – Pardis
    Jun 11, 2012 at 5:45

2 Answers 2

Reset to default
6
$\begingroup$

"Generalization" refers to the ability of a classifier to correctly classify instances that it has not yet seen as part of its training and is always a desirable feature. The opposite of generalization would be over-fitting, which you always want to avoid (Although a classifier might also not generalize well if the training domain is different from the domain it will finally be applied to, which might not be due to over-fitting in the strictest sense).

The terms recall and precision have very specific meanings, while your use of "general" and "specific" are not standard terminology I don't think. For binary classifiers, it is customary to refer to one class of instances as "positive" and the other as "negative". For example, if you are trying to build a classifier that recognizes pictures with faces in them, then you might define the class of all pictures that contain faces as the positive class. The term recall is then defined as the proportion of all positive instances that were classified correctly, while precision is the proportion of all instances classified as positive that were actually positive.

Here is an example. Lets say of $40$ pictures, $30$ contain faces. If a classifier classifies $25$ pictures as positive, of which $20$ actually do contain faces, then the recall is $\frac{20}{30}=2/3$ while precision is $\frac{20}{25}=0.8$.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks! Then, would it be correct to say that learning features "with high class discrimination power" improve the "generalization capability" of a classifier? Can we refer to such features as "general" features ? $\endgroup$
    – kanzen_master
    Jun 13, 2012 at 5:42
  • 1
    $\begingroup$ using good features is always key to good classifier performance, but whether or not a specific feature helps in "generalizing" the classifier is not clear without context. You need to perform tests on the classifier to ensure that it works well in the intended domain and that it has not been overfitted to the training set. One way overfitting might occur is if you are choosing among a very large set of possible features. some features may seem to work very well in the training set, but this could be purely random noise. Therefore you need to test the generalizability of your classifier. $\endgroup$
    – ALiX
    Jun 14, 2012 at 20:09
  • $\begingroup$ Thanks. I understand that the class discrimination power of a feature heavily depends on the context. However, In my case (I am dealing with sentiment analysis), some features like "happy" or "nice" (1) are likely to be better features than "map" or "phone"(2). So, my question is, although it all depends on the test dataset, if we prioritize features like (1) rather than (2) are we likely to end up with a classifier with higher recall ? $\endgroup$
    – kanzen_master
    Jun 15, 2012 at 2:56
  • $\begingroup$ In short yes. this falls under the term "feature selection", which can be done manually, computationally, or both. Prior knowledge about your domain of interest should definitely help in creating better classifiers. But this can of course be tested as well. $\endgroup$
    – ALiX
    Jun 15, 2012 at 3:47
  • $\begingroup$ Thanks again. I see. The problem I am facing now is target-independent sentiment analysis, which means that there is not a particular domain I am focusing on. This is why I am struggling now to get the most "general" sentiment features (the ones that could be more domain-independent than others) $\endgroup$
    – kanzen_master
    Jun 15, 2012 at 5:42
2
$\begingroup$

Answering your 3rd question. Your training data should be as representative of the domain you are trying to learn as possible. The learned classifier should then be used in this same domain.

If you want to make use of the things you have learned in one domain and you think that this might be applied in another but somehow related domain then you are talking about 'transfer learning'. For example, a table classifier might have reusable parts if you want to learn a chair classifier.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the name. Didn't know it was called "transfer learning" $\endgroup$
    – kanzen_master
    Jun 13, 2012 at 5:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.