1
$\begingroup$

I know of four possible reasons:

  • overfitting
  • underfitting
  • input data doesn't represent the problem (which I guess is underfitting)
  • classifier isn't suitable (e.g. problem is not linear)

Are there any other reasons why a classifier produces bad results? When should one try a different model? Could you please point me out to some good literature which I get read?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ What do you mean by "bad" results? $\endgroup$
    – Tim
    Commented Nov 3, 2015 at 11:26
  • $\begingroup$ @Tim By bad results I mean producing wrong predictions regarding a problem. What are other meanings of "bad" results? $\endgroup$
    – DerTom
    Commented Nov 3, 2015 at 11:29
  • $\begingroup$ @Tim I see, well in this matter I am looking at all different kind of models and the goal is to achieve the best possible prediction. $\endgroup$
    – DerTom
    Commented Nov 3, 2015 at 12:06
  • 2
    $\begingroup$ To @tim's point, "bad" is a subjective and nontechnical term. This paper from the Jour ML Research Do we Need Hundreds of Classiers to Solve Real World Classication Problems? reviews and compares a boatload of methods with the goal of objectively identifying the "best" performing algorithm. Random Forests won. jmlr.org/papers/volume15/delgado14a/delgado14a.pdf $\endgroup$
    – user78229
    Commented Nov 3, 2015 at 13:57
  • $\begingroup$ Interesting. I might add "the input data is unmodelable", i.e. it is nothing but random noise. The definition of "bad" is worth examining too, of course. $\endgroup$
    – Andrew Charneski
    Commented Nov 3, 2015 at 16:12

1 Answer 1

Reset to default
1
$\begingroup$

You either fail to capture patterns that explain the outcome (underfitting) or fit to coincidences that will not be present in new data (overfitting). All four bullet points correspond to one of these.

The first two are obvious. The third could be either overfitting or underfitting. It would be underfitting if you simply do not measure an important determinant of the outcome of interest, and it would be overfitting if you have biased data that lead to a model that only applies to a subset of the population where you would want to apply the model (e.g., training using data collected from children, applying to children and adults). The fourth is another example of underfitting. Plenty of models allow you to model nonlinear boundaries between groups. If you force the boundary to be a line when it should be curved, you have underfit.

Note that you can simultaneously overfit and underfit.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.