3
$\begingroup$

I've trained a classifier model using Vowpal Wabbit to decide if a person is Male or Female based on name alone. I assigned labels Male=0 and Female=1. When I ran Vowpal Wabbit in prediction mode, the output had values varying between 0.0 and 1.0. I interpreted this as p <= 0.5 ==> Male and p > 0.5 ==> Female. With this the accuracy of prediction was 85%.

I'm now trying to train a classifier model for another problem where there are 200 labels instead of just two like above. The labels go from 0 to 199.

The prediction output has values between 0.0 and 199.0. When I get a value such as 99.56 (for example), how do I interpret this? Does it map to label 100?

$\endgroup$

1 Answer 1

3
$\begingroup$

I've come to understand this in the following terms:

There are two modes of operation for classification.

1) Binary classification 2) Multi-class classification

In Vowpal Wabbit, Multi-class classification is implemented as a learning reduction mechanism using Binary classification.

I view the binary classification output value (ranging between 0.0 and 1.0) as a confidence level for the label assigned the index 1.

Eg: if male = 1 and female = 0, any value >= 0.5 represents a higher confidence that the item being classified is male than female.

I was using Vowpal wabbit incorrectly when I tried to fit a multi-class problem into this style of operation and believe the results I got are meaningless. You can explore using the learning reduction method called "One-Against-All" exposed via that option --oaa in the vw tool. This will make predictions which map to the labels specified by you.

Eg: if the classes are red=0, blue=1, green=2 the predictions will no longer be floating point values but map exactly to 0, 1 and 2.

HTH

$\endgroup$
1
  • $\begingroup$ My understanding of ML terminology is rudimentary at best. Please feel free to correct. $\endgroup$ Apr 3, 2014 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.