I'm trying to build a one vs all multi-class classifier in R and my results are pretty poor. I'm curious if anyone can suggest a piece of example code somewhere on the internet.

I'm trying to classify some text. There are 5 classes that the text strings could fall into. When I search for 'one vs all classifier' I get pages of papers on how great they are but can't find a single implementation.


closed as not a real question by cardinal, gung, whuber Aug 14 '12 at 20:28

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    $\begingroup$ I would suggest to add some more information on what you are trying to do. E.g., what are the type and number of predictors, what is the sample size, what does it mean that "results are pretty poor." Otherwise, I'm afraid this question is barely related to statistical analysis as defined in our FAQ. $\endgroup$ – chl Jul 25 '12 at 9:51

Rather than one-versus-all, you may want to try methods that deal with multi-class outcomes directly: multinomial regression (glmnet package in R), the pamr package (shrunken centroids), random forests (randomForest) to name a few.

I don't know of any packages that implement one-versus-all for you, but it is simple enough to code. For example, with leave-one-out cross validation, we would:

hold out one observation

fit all possible binary models (class 5 versus 1,2,3,4)

the model must output a score (such as a predicted probability)

obtain all scores for the models applied to the held out sample

choose the class with highest score as the predicted class

I find this procedure awkward, but it seems to be used often, and it is also mentioned briefly in Hastie (Element of Statistical Learning) with respect to support vector machines.

  • $\begingroup$ Thank you for the suggestions. I am currently working with many of those methods. However, I need a working example of OVA. The main reason is my mediocre coding skills especially when more than one loop is involved. $\endgroup$ – screechOwl Jul 26 '12 at 12:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.