18
$\begingroup$

I have a space of 35 dimensions (attributes). My analytic problem is a simple classification one.

Out of 35 dimensions, more than 25 are categorical and each attribute takes more than 50+ types of values.

In that scenario, introducing a dummy variable also will not work for me.

How can I run an SVM on a space which has a lot of categorical attributes?

$\endgroup$
22
$\begingroup$
  1. If you are sure the categorical attribute is actually ordinal, then just treat it as numerical attribute.
  2. If not, use some coding trick to turn it into numerical attribute. According to the suggestion by the author of libsvm, one can simply use 1-of-K coding. For instance, suppose a 1-dimensional category attribute taking value from $\{A,B,C\}$. Just turn it into 3-dimensional numbers such that $A = (1,0,0)$, $B = (0,1,0)$, $C = (0,0,1)$. Of course, this will incur significantly additional dimensions in your problem, but I think that is not a serious problem for modern SVM solver (no matter Linear type or Kernel type you adopt).
$\endgroup$
  • $\begingroup$ +1 This is what I was going to say, as well! As an aside, I'd also add that recursive partitioning is sometimes used to identify where to best make cuts in continuously-valued features, to partition them into bins. $\endgroup$ – Kyle. Mar 28 '13 at 3:48
  • $\begingroup$ Interesting! The "recursive partitioning" sounds a (binary) tree to me. Any difference between these two ideas? Besides, SVM is already able to deal with continous feature, why shall we turn it to bins (again, categorical data) ? $\endgroup$ – pengsun.thu Apr 1 '13 at 15:06
  • 3
    $\begingroup$ Isn't creating k-1 dummy variables enough for a k level categorical variable? eg A=(1,0,0), B=(0,1,0) here, skipping (0,0,1)? $\endgroup$ – Outlier Jun 9 '14 at 5:12
  • $\begingroup$ follow up question: is no additional scaling required of the dummied 0-1 data? $\endgroup$ – AZhao Feb 9 '18 at 3:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.