How Rapidminer handle same distance for KNN Algorithm

Question

Actually I already asked in rapidminer forum, but no one has given an answer yet.. https://community.rapidminer.com/discussion/55963/how-k-nn-algorithms-work-with-same-distance-in-rapidminer#latest

I can't find a satisfying answer for KNN-algorithm with same euclidean distance in rapidminer..

I found a similar question, but it's not explain how rapidminer work for this case. Dealing with ties, weights and voting in kNN

say k=5. Now I try to classify an unknown object by getting its 5 nearest neighbours. What to do, if distance is a lot of the same distance.. if after determining the 4 nearest neighbors, the next 2 (or more) nearest objects have the same distance and diferent label? Which object of these 2 or more rapidminer chosen as the 5th nearest neighbor?

I confused.. I try in excel, and the result is diferent with rapidminer for some data. in excel the result label is "LU":

but the result in rapidminer is "LT" :

result rapidminer weighted vote is checked is "LU" :

How rapidminer work with case like that... how rapidminer sorting the distance ?... something wrong with my data ?, or rapidminer sorting random for same distance?

thanks you in advance for your help

Answer 1

I answered your original question on StackOverflow this way:

In these cases, where distances are the same, RapidMiner kNN uses the internal sorting of the ExampleSet that was used at training time. So internally it picks the examples that it "saw first".

Try to change the sorting before building the kNN model, it should give different results.

You can verify it with the official source code on github: https://github.com/rapidminer/rapidminer-studio/blob/master/src/main/java/com/rapidminer/operator/learner/lazy/KNNClassificationModel.java

Answer 2

A solution if you are concerned with ties and worried about what kind of tie breaking method a particular algorithm might use is to add a very small random value to one of your predictors (if possible). This way ties will be broken randomly, and won’t depend on the order of your input set