# Questions tagged [k-nearest-neighbour]

A non-parametric method of classification and regression. The input consists of the $k$ closest training examples in the feature space. The output is either the mode of the neighbors (in classification) or their mean (in regression).

573 questions
Filter by
Sorted by
Tagged with
5 views

### find k nearest neighbors of a given data point x in KNN_classifier [migrated]

in python, the knn algorithm can be applied on training data with the code like below: ...
• 133
55 views

### Modelling spatial autocorrelation with GAMs

the topic of spatial autocorrelation (SA) within the context of generalized additive models has already been discussed in several posts within this forum, see e.g. Why does including latitude and ...
20 views

### Weighting (rescaling) the dimensions (features) in $k$ nearest neighbors

When using $k$ nearest neighbors,one have to normalize the data set ( if X is the matrix of data such that every row represents a feature or dimension and every column represents a sample point, then ...
38 views

### Are there strategies for measuring accuracy of Euclidean distance-based similarity without ground truthing?

I have subjects with about 200 features each. These feature vectors are stored in a vector database, where similarity searching with Euclidean distance is used to find subjects that are similar to a ...
• 23
1 vote
29 views

### I was working on this machine learning question and I dont seem to get how to solve it! [closed]

I have tried doing it so many ways, but none seem to work. I don't understand these LOOCV errors
• 11
1 vote
38 views

### KNN K = 1 Training on itself vs K > 1

When training a $KNN$ algorithm, why is that with $K = 1$, the model trains using the "1 nearest observation to each training point" and treats this as itself resulting in a training error ...
• 161
20 views

### Weighted KNN imputation

Consider the following piece of python code. ...
• 1,443
47 views

### KNN-regression vs Linear regression

Is there any assumption on data or any number of k, that makes kNN-regression equivalent to linear regression?
• 87
9 views

### Handling valid missing data in prediction model features

I'm developing an elastic net model using caret, with k-nearest neighbours to handle missing data in the features. Some features are conditional on others such that a missing value is valid e.g., ...
63 views

### The error-rate in "The elements of statistical learning"

This picture is from the book "the elements of statistical learning": I am wondering how the test-error rate is calculated based on how the describe the simulation at the start? How do they ...
• 131
14 views

### Is there some sort of multilevel KNN/ML model I can use to figure out which users will buy specific products?

I am wondering if there is some sort of multilevel model that I can use to identify likely buyers of specific products or create a lookalike audience. The issue is that I have 1000s of products and ...
• 65
19 views

### Feature Selection Before or After kNN Imputation?

From my understanding, kNN imputation is dependent on the variables where any two cases do not have missing values. Thus, would it be ideal to do feature selection before or after imputation?
20 views

1 vote
123 views

### Elbow method for tuning DBSCAN when the minimum number of points per cluster is one

The elbow method calls for setting the number of nearest neighbors (let's call it $k$) to the minimum number of points for a cluster (let's call it $m$), but what do you do when $m\leftarrow1$? Is the ...
101 views

### Why is the space complexity of KNN $O(dN)$?

If you Google search the space complextiy of KNN, virtually all answers are saying that it costs $O(dN)$, where $d$ is the dimension of our data and $N$ is the number of our data. But why? I ...
• 1,580
66 views

### How is the "training error" of KNN plotted?

On https://www.cs.ubc.ca/~murphyk/Teaching/CS340-Fall07/L4_knn.pdf Page 6 The author (Kevin Murphy) plots the training and test errors associated with kNN classifer. I am not sure how training error ...
• 1,580
25 views

### How do I select a KNN model?

I am considering methods of selecting an optimal K Nearest Neighbors model for classifying a pixel as representing a refugee's blue tarp or not. See https://www.kaggle.com/datasets/billbasener/pixel-...
• 121
1 vote
53 views

### Random Forest Variable Importance as a way to weight Nearest Neighbor Variables

I'm employing a nearest neighbor algorithm to find a real NFL game that is the most similar to my projected stats. Not all statistics have the same predictive-importance when projecting the outcome of ...
1 vote
85 views

### Is K-Nearest Neighbor and Nearest Neighbor algorithm the same?

Does anyone know if there is a difference between K nearest neighbor (KNN) and nearest neighbor algorithm (NN)? And if they are how are they different? So far all I know is that NN is unsupervised ...
51 views

### Multiple equidistant neighbours in 1-nearest-neighbour - how to break ties?

I am wondering what should happen when trying to compute leave-one-out cross-validation error when there are multiple 1-nearest-neighbours that is equidistant to the training point. As an example, ...
• 21
1 vote
43 views

### Steps for condensed nearest-neighbor algorithm

I'm in the middle of writing my Master's thesis on undersampling techniques in imbalanced datasets, and I wanted to refer on this paper explaining the Condensed Nearest-Neighbor algorithm and what ...
• 11
34 views

### what does the k-NN Algo do with equidistant training points from the test point?

This was my professor's interpretation but he didn’t provide an example: there could be training points at the same distance from x such that more than k points are closest to x. In this case, we ...
• 21
23 views

### My KNN result not as expected for all numerical data

I have developed the following KNN code and tested it against several datasets with >90% accuracy (the wdbc dataset from UCI for one, granted it was a categorical result), however when using the ...
3k views

### How to adjust the classification thresholds in a multiclass classification problem?

I am facing a multiclass classification problem where I have 4 classes and one of them dominates over the others. I use a KNN classification model and the majority of the instances are being ...
• 141
1 vote
26 views

### How to use Graph Edit Distances and the graph-level features at the same time

I have a distance matrix between paris of graphs computed by Graph Edit Distances. Besides, I have also a group or class label for each graph. Besides, each graph is assigned a target value in real ...
• 11
165 views

### What is the pseudocode for the fastest possible k-nearest-neighbors (KNN) algorithm? [closed]

I have a BERT model that's fine-tuned so that given a sentence in my X column, the model gives a vector that approximates the corresponding sentence in my multidimensional Y array. I'd like to use the ...
• 343
150 views

### How to interpret NearestNeighbor results obtained using cosine similarity for tf-idf vectors

Why is the top result obtained using cosine similarity extremely close to 0 not the expected 1? That implies complete orthogonality. Data: 100k documents/rows with 2000 features(TF_IDF values of ...
• 111
546 views

### Alternative to KNN

I have been working with a custom classification model with some success, and I was wondering if there is a package in Python or R does the same thing. Regular KNN chooses a K and does classification ...
• 149
56 views

### KNN does not take into account the actual magnitude of the distance

Suppose I have 3 two dimensional points $(0, 0), (10000000, 0), (-1, 0)$ and $(0, 0)$ has label 1.0. If we were to use 1-NN to predict the label for $(10000000, 0), (-1, 0)$, then the answer for both ...
• 195
1 vote
266 views

### What's a Good $R^2$ Score in K Nearest Neighbors?

I'm fitting SciKit-Learn's KNeighborsRegressor on a 5 dimensional space and my model performance is peaking at a score of $\sim 0$. In their documentation they say ...
• 645
62 views

### How to find or choose the value of k in KNN?

I have a doubt regarding how to choose the value for k in KNN. I saw in many websites to take sqrt of samples. Is the sample here total number of rows or (number of rows x number of columns)? Can ...
203 views

### Dealing with near duplicates of the same person in K Nearest Neighbor algorithm

For context, I am a beginner and this is my first time attempting to implement a machine-learning algorithm. This is for school. I am attempting to predict whether a 100-meter dash athlete wins a ...
3k views

### Accuracy score of my KNN model is constant as k increases?

This is my first time using a machine learning algorithm. It is for a school project. My model is attempting to predict from inputs: Weight, Height, and Age whether an Olympic athlete (100-meter dash) ...
1 vote
24 views

### why does SVM outperforms KNN in 1-gram but in 2,3,4, and 5 KNN outperforms SVM?

my project is authorship attribution which is a multiclass classification, the number of classes is 150, and the number of documents is 2798; it is also an unbalanced issue some classes have more ...
449 views

### Clustering by using Locality sensitive hashing *after* Random projection

It is well known that Random Projection (RP) is tightly linked to Locality Sensitive Hashing (LSH). My goal is to cluster a large number of points lying in a $d$-dimensional Euclidean space, where $d$ ...
326 views

### kNN Classifier Asymptotic Error Rate versus Bayes Error Rate

Suppose we are in the realm of $M$ class classification, $M \in \mathbb{N}$. I have seen the following result stated many times, but only proven for the case $k = 1$. I would like to prove it for ...
• 183
221 views

### Example of KNN overfitting with k=1

I know that with k=1 a KNN lead to overfitting, this is because it follows the noisy data of the training sample and not generalize well on new input sample. But I am confused on how this happens, I ...
1 vote
46 views

### What is the best way (in terms of performance and accuracy) to perform value estimation/prediction?

How do I predict the value of an item given its features and attributes? What is the best approach to this regression problem, in terms of performance as well as accuracy? And how do I process this ...
• 111