I want build a classifier that classifies sentences into two categories, and for that I have a training set of 1000 labeled sentences. My features consist of a list of about 8000 words, and for each sentence I measure the word frequency (feature divided by total words in the sentence).

I've been reading up on the Mahalanobis distance, but I haven't yet managed to understand how I could apply it to my problem. How can I use it to select the features that work best for distinguishing between the two categories?

Thank you!


You can use the Mahalanobis distance to quantify the distance of features of sentence $S_1$ to sentence $S_2$. Using this distance $d(S_1,S_2)$, you can apply the k-means algorithm that is suitable also for classification. For k-means classification, see e.g. this.

Alternatively, you can apply a kernel function, i.e. $K(d(S_1,S_2))$ to this distance and learn an RBF network that would classify your sentences. To be more specific, the approach can be described in the following steps:

  1. Select some (say 50) sentences from your set at random. Let us call them centers.
  2. For all 1000 sentences calculate distances to the centers. Thus, you will have a 1000 x 50 distances.
  3. Quantify the values of the kernels. That means, apply a kernel to each of those 1000 x 50 numbers.
  4. Use a linear regression to fit the output value (index of the class either 0 or 1).
| cite | improve this answer | |
  • $\begingroup$ Thank you! Do I have to do this for every pair of sentences? $\endgroup$ – user84206 Aug 5 '15 at 13:37
  • $\begingroup$ Tried to be clearer. What is the programming language you use? $\endgroup$ – Karel Macek Aug 5 '15 at 13:42
  • $\begingroup$ I'm using MATLAB for this, it's part of an assignment. I like the first approach, I will try to implement it. $\endgroup$ – user84206 Aug 5 '15 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.