I was working with SVR, and wondering, why can't I solve a natural regression problem as a multiclassification task ?

Example: I have for a regression problem: targets 1, 5 and 10, trying to fit Integers numbers inside of this range.

Why can't I use a multiclassification with targets 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ?

What are the advantages and disadvantages of such approach ?

  • One of disadvantages that I can see, is that I must have a complete target training set (1 to 10) at the multiclass problem. Any other problems ?

Some issues that come in mind:

  • In classification you don't take into account the relations between classes. In you example, labels 1 and 10 would be treated equally while in reality they are "far" from each other. That creates problems in training and in assessment of performance ( classifying a sample as "2" while it is a "1" shouldn't be the same as classifying it as "10".
  • Typically in regression you don't have 10 distinct values as in your example, but the whole range of real numbers within some range. That is you have infinite number of "classes".
  • $\begingroup$ That was a really good perception on the problem. But, there are advantages ? $\endgroup$ Aug 8 '14 at 1:24

The main issue is that you lose information about the order, which is quite important.

Imagine that you do classification and your classifier somehow has 5 premises about $y$, 2 that $y=a$, 2 that $y=b$ and 1 that $y=c$ -- this looks pretty bad, that $y$ is either $a$ or $b$ but we don't really know.
Now, let's say we have the same for regression and $a=6$, $b=7$ and $c=8$; now those premises seem consistent and pretty clearly suggests that $y=7$.

Also the methods that optimize error can get a better assessment knowing that misclassifying $3$ with $4$ is not as bad as $1$ with $9$.

  • $\begingroup$ Sequencing is a really important thing to consider, you are right. Can you see any advantages ? $\endgroup$ Aug 8 '14 at 1:25
  • $\begingroup$ The only I can think of is that some methods simply can't perform regression and we can force them to do it this way. $\endgroup$
    – user88
    Aug 8 '14 at 1:27
  • $\begingroup$ mbq, I'm working with SVR, and tried to multiclass it with SVM, beside this, I only work with ANNs. Both SVR and ANNs provides fitting. Do you have any example that cannot perform regression ? $\endgroup$ Aug 8 '14 at 1:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.