1
$\begingroup$

I want to classify histograms/distributions using K-Nearest-Neighbor. I can measure distances/dissimilarities between the distributions (using euclidean distance, kullback-leibler divergence...), thus I can obtain distance matrices. I was wondering since Nearest Neighbors measure distances anyway, can I incorporate distance matrices directly into the algorithm?

Also if you know a function in R or python that already exists, I'm interested. thank you

More details on my dataset: I have more than 100 observations that I want to classify in 2 classes (I have the labels) and all the features (4 features) are histograms (1 feature = 1 histogram).

UPDATE:

Using R: function "knn_dist" from "evclust" package

Improve this question
$\endgroup$
7
  • $\begingroup$ Are you looking for something like K-means? $\endgroup$ – user2974951 Nov 30 '18 at 10:34
  • $\begingroup$ Not really, I know that you can use similarity matrix as an input in K-medoid algorithm but since I have class labels I want to use a supervised learning for the classification task (k-means and K-medoids are unsupervised learning but KNN is supervised learning) $\endgroup$ – learneRS Nov 30 '18 at 11:39
  • 1
    $\begingroup$ Read this thread about k-means stats.stackexchange.com/q/32925/3277. Same is true for other analyses. To embed a distance matrix into feature space by means of MDS with sufficient number of dimensions giving good fit (stress value). Then proceed with knn or k-means etc. as usual. Of course, your matrix cannot be huge, or MDS won't cope. $\endgroup$ – ttnphns Dec 1 '18 at 9:42
  • $\begingroup$ Thank you @ttnphns .So if I understand well, you can't just use a distance matrix as an input in knn algorithm , you have to do MDS first than use knn like usual ? and second question MDS works only if the distance we use is the euclidean distance? $\endgroup$ – learneRS Dec 2 '18 at 18:12
  • 1
    $\begingroup$ MDS is for any distance. I recommend you to read something about MDS before using it. $\endgroup$ – ttnphns Dec 3 '18 at 7:41
1
$\begingroup$

yes, it's possible because KNN finds the nearest neighbor, you already have distance/similarity matrix then the next step is to fix k value and then find the nearest value. Out of all the nearest neighbor take the majority vote and then check which class label it belongs.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.