I would like to predict the category of the provided data by using K-NN algorithm. Here is an example of the training data set

category sex      age
1        male     19-20-21
5        female   40-41-42
x        male     18        <- provided data

the features of the training data set could be numbers such as age but especially additionally text data. How could you suggest me to calculate distance since I have not data numbers but text data. By using hamming distance perhaps?


1 Answer 1


The minimal approach for text based features is to used bag of words. Basically the whole data set allows you to define the vocabulary of term used and then each item has a 1/0 value for the presence/absence of each term. This lead to pretty large sparse vector of 0/1 but it's then easy to compute distance using any kind of cosine similarity approach.

To improve over the bag of words, you can refine the 0/1 using term frequency (or even better tf/idf).

At the end, you will still need to aggregate the distance computed over the text data with the other features. Since the range and dynamic might be very different over these feature distances, it might be tricky. Try to start with basic linear combination (weighted sum). If not to clumsy, the weights night be easy to fix. If not, then you can try optimization approach to approximate good values.

  • $\begingroup$ I have a question. In the example above I have only two features. What about when I have more features let suppose more than twenty. Is it still possible to use this algorithm? $\endgroup$
    – Mazzy
    Commented Jul 24, 2014 at 9:11
  • $\begingroup$ Aren't confusing "date fields" with "features"? If I understand well, you have few text fields that will be converted into features using bag of words. If you have more text fields, you can choose to mix them or have separated bags of words. Then for the combination, of course it can be extended to more features. You will have more weights to define which might be longer but sometimes, basic values (every weights at 1) just to the trick. $\endgroup$
    – gdupont
    Commented Jul 29, 2014 at 5:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.