Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a dataset like

| income | year |        use        |
|  46328 | 1989 | COMMERCIAL EXEMPT |
|  75469 | 1998 | CONDOMINIUM       |
|  49250 | 1950 | SINGLE FAMILY     |
|  82354 | 2001 | SINGLE FAMILY     |
|  88281 | 1985 | SHOP & HOUSE      |

I embed it into a LIBSVM format vector space

+1 1:46328 2:1989 3:1
-1 1:75469 2:1998 4:1
+1 1:49250 2:1950 5:1
-1 1:82354 2:2001 5:1
+1 1:88281 2:1985 6:1

Feature indices:

  • 1 is "income"
  • 2 is "year"
  • 3 is "use/COMMERCIAL EXEMPT"
  • 4 is "use/CONDOMINIUM"
  • 5 is "use/SINGLE FAMILY"
  • 6 is "use/SHOP & HOUSE"

Is it OK to train a support vector machine (SVM) with a mix of continuous (year, income) and categorical (use) data like this?

share|improve this question
You should spell out "SVM", at least once. – Peter Flom Feb 21 '13 at 1:28
Make sure you scale that data! – Patrick Caldon Feb 21 '13 at 2:16
up vote 5 down vote accepted

Yes! But maybe not in the way you mean. In my research I frequently create categorical features from continuously-valued ones using an algorithm like recursive partitioning. I usually use this approach with the SVMLight implementation of support vector machines, but I've used it with LibSVM as well. You'll need to be sure you assign your partitioned categorical features to a specific place in your feature vector during training and classification, otherwise your model is going to end up jumbly.

Edit: That is to say, when I've done this, I assign the first n elements of the vector to the binary values associated with the output of recursive partitioning. In binary feature modeling, you just have a giant vector of 0's and 1's, so everything looks the same to the model, unless you explicitly indicate where different features are. This is probably overly specific, as I imagine most SVM implementations will do this on their own, but, if you like to program your own, it might be something to think about!

share|improve this answer
thanks Kyle, can you be a little more specific? What do you mean "assign your partitioned categorical features to a specific place"? – Seamus Abshere Feb 21 '13 at 2:42
@SeamusAbshere No problem! I edited my answer to address this! – Kyle. Feb 21 '13 at 3:01
I feel like I've heard that libsvm does what you're talking about automatically - any thoughts? – Seamus Abshere Feb 21 '13 at 15:31
@SeamusAbshere I imagine you're right, but I don't know for sure. Now that I think about it, I'm not sure how it could work any other way. – Kyle. Feb 21 '13 at 15:51
Emboldened by @Kyle's answer, I wrote a Ruby library (VectorEmbed) that does this conversion (embedding) automatically, both for categorical (using Murmur32 hashes) and continuous data. It outputs libsvm-formatted files. – Seamus Abshere Mar 29 '13 at 0:38

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.