# Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what label the incorrect prediction should output.

But suppose I want to predict a score from 1 to 5, and I prefer to err closer to the truth, how do I go about it then? $\lt k$ vs $\ge k$ comes to mind, because in this case labels which are closer together will have larger training data overlap, but is it theoretically sound? Specifically, is overall performance likely to suffer? More generally, what are state-of-the-art techniques I could employ in this situation?

• Check out structured SVM. You can find an implementation here. – Marc Claesen Feb 25 '14 at 17:22
• @MarcClaesen I can't see how it relates to what I'm trying to do here. – Alexei Averchenko Feb 26 '14 at 10:12
• Unless I misunderstood, your class labels are structured, e.g. $5$ implies $\geq 4$ ... $\geq 1$. This can be modeled directly in structured SVM. – Marc Claesen Feb 26 '14 at 10:30
• So you want me to output the whole chain of numbers? But this is just making labels different, it doesn't introduce any dependence between them, how is this going to help me? – Alexei Averchenko Feb 26 '14 at 10:43
• Why isn't this just a regression problem? Train a regressor and round to the nearest number in your range? – MotiN Aug 22 '18 at 18:25