Questions tagged [k-nearest-neighbour]
A non-parametric method of classification and regression. The input consists of the $k$ closest training examples in the feature space. The output is either the mode of the neighbors (in classification) or their mean (in regression).
556
questions
0
votes
0
answers
27
views
Distance metric for sparse matrix with imbalanced data
I am trying to find similar movies in a dataset where each movie has zero to five genres. (I also have more dimensions but those are not relevant to the question.)
For this I'm trying to use nearest ...
- 101
0
votes
0
answers
10
views
For KNN: How to justify that the probability of points to fall into a sphere of volume $V$ is $p(X)V$
https://www.cs.cmu.edu/~lwehbe/10701_S19/files/Lecture_3.pdf
At the end of these notes, there is a short paragraph.
Let $x$ be a test point. Let $x_1, \ldots, x_K$ be its $K$ nearest
neighbors. Let $...
- 651
1
vote
1
answer
94
views
Elbow method for tuning DBSCAN when the minimum number of points per cluster is one
The elbow method calls for setting the number of nearest neighbors (let's call it $k$) to the minimum number of points for a cluster (let's call it $m$), but what do you do when $m\leftarrow1$? Is the ...
- 11
5
votes
1
answer
50
views
Why is the space complexity of KNN $O(dN)$?
If you Google search the space complextiy of KNN, virtually all answers are saying that it costs $O(dN)$, where $d$ is the dimension of our data and $N$ is the number of our data.
But why?
I ...
- 1,460
0
votes
0
answers
92
views
Using SPSS Propensity Score Matching yields Propensity Scores but no Match IDs
I'm using the IBM provided PSM module in SPSS 26 on MacOS.
I have 1 group indicator and 4 covariates (3 scalar-1 nominal).
I tried match tolerances of 0.5 and 1.
I tried with and without replacement ...
3
votes
1
answer
33
views
How is the "training error" of KNN plotted?
On https://www.cs.ubc.ca/~murphyk/Teaching/CS340-Fall07/L4_knn.pdf
Page 6
The author (Kevin Murphy) plots the training and test errors associated with kNN classifer.
I am not sure how training error ...
- 1,460
0
votes
1
answer
23
views
How do I select a KNN model?
I am considering methods of selecting an optimal K Nearest Neighbors model for classifying a pixel as representing a refugee's blue tarp or not. See https://www.kaggle.com/datasets/billbasener/pixel-...
- 121
1
vote
0
answers
35
views
Random Forest Variable Importance as a way to weight Nearest Neighbor Variables
I'm employing a nearest neighbor algorithm to find a real NFL game that is the most similar to my projected stats. Not all statistics have the same predictive-importance when projecting the outcome of ...
1
vote
1
answer
53
views
Is K-Nearest Neighbor and Nearest Neighbor algorithm the same?
Does anyone know if there is a difference between K nearest neighbor (KNN) and nearest neighbor algorithm (NN)? And if they are how are they different?
So far all I know is that NN is unsupervised ...
user379637
0
votes
0
answers
41
views
How can I interpret KNN clustering using FNN package in R for analyzing football passes?
I have a dataframe of 411 rows where each row represents a pass (football) and four columns. Two columns are the x and y coordinates of the location where the pass starts, and the other two columns ...
2
votes
1
answer
33
views
Multiple equidistant neighbours in 1-nearest-neighbour - how to break ties?
I am wondering what should happen when trying to compute leave-one-out cross-validation error when there are multiple 1-nearest-neighbours that is equidistant to the training point.
As an example, ...
- 21
1
vote
0
answers
25
views
Steps for condensed nearest-neighbor algorithm
I'm in the middle of writing my Master's thesis on undersampling techniques in imbalanced datasets, and I wanted to refer on this paper explaining the Condensed Nearest-Neighbor algorithm and what ...
- 11
2
votes
1
answer
20
views
what does the k-NN Algo do with equidistant training points from the test point?
This was my professor's interpretation but he didn’t provide an example:
there could be training points at the same distance from x such that more than k points are closest to x. In this case, we ...
- 21
0
votes
0
answers
20
views
My KNN result not as expected for all numerical data
I have developed the following KNN code and tested it against several datasets with >90% accuracy (the wdbc dataset from UCI for one, granted it was a categorical result), however when using the ...
- 1
3
votes
2
answers
1k
views
How to adjust the classification thresholds in a multiclass classification problem?
I am facing a multiclass classification problem where I have 4 classes and one of them dominates over the others. I use a KNN classification model and the majority of the instances are being ...
- 31
1
vote
0
answers
26
views
How to use Graph Edit Distances and the graph-level features at the same time
I have a distance matrix between paris of graphs computed by Graph Edit Distances. Besides, I have also a group or class label for each graph. Besides, each graph is assigned a target value in real ...
- 11
0
votes
1
answer
132
views
What is the pseudocode for the fastest possible k-nearest-neighbors (KNN) algorithm? [closed]
I have a BERT model that's fine-tuned so that given a sentence in my X column, the model gives a vector that approximates the corresponding sentence in my multidimensional Y array.
I'd like to use the ...
- 343
0
votes
1
answer
109
views
How to interpret NearestNeighbor results obtained using cosine similarity for tf-idf vectors
Why is the top result obtained using cosine similarity extremely close to 0 not the expected 1?
That implies complete orthogonality.
Data:
100k documents/rows with 2000 features(TF_IDF values of ...
- 111
0
votes
0
answers
50
views
Why LOOCV is easier to compute for kNN?
On the book Artificial Intelligence - A modern approach there is a paragraph stating the following:
Most nonparametric models have the advantage that it is easy to do leave-one-out cross-validation ...
- 101
0
votes
0
answers
355
views
Alternative to KNN
I have been working with a custom classification model with some success, and I was wondering if there is a package in Python or R does the same thing. Regular KNN chooses a K and does classification ...
- 139
3
votes
1
answer
51
views
KNN does not take into account the actual magnitude of the distance
Suppose I have 3 two dimensional points $(0, 0), (10000000, 0), (-1, 0)$ and $(0, 0)$ has label 1.0. If we were to use 1-NN to predict the label for $(10000000, 0), (-1, 0)$, then the answer for both ...
- 165
1
vote
1
answer
117
views
What's a Good $R^2$ Score in K Nearest Neighbors?
I'm fitting SciKit-Learn's KNeighborsRegressor on a 5 dimensional space and my model performance is peaking at a score of $\sim 0$.
In their documentation they say ...
- 461
0
votes
1
answer
55
views
How to find or choose the value of k in KNN?
I have a doubt regarding how to choose the value for k in KNN. I saw in many websites to take sqrt of samples. Is the sample here total number of rows or (number of rows x number of columns)? Can ...
- 1
0
votes
0
answers
121
views
Implementing NESTED/DOUBLE k-fold cross validation in sklearn
Is the following the right approach for implementing nested-k fold cross validation using sklearn library? As far as I know, we create k splits of training/test set, and each training set resulting ...
- 115
2
votes
1
answer
143
views
Dealing with near duplicates of the same person in K Nearest Neighbor algorithm
For context, I am a beginner and this is my first time attempting to implement a machine-learning algorithm. This is for school. I am attempting to predict whether a 100-meter dash athlete wins a ...
2
votes
2
answers
2k
views
Accuracy score of my KNN model is constant as k increases?
This is my first time using a machine learning algorithm. It is for a school project. My model is attempting to predict from inputs: Weight, Height, and Age whether an Olympic athlete (100-meter dash) ...
0
votes
0
answers
171
views
SPSS: how to calculate nearest neighbor from k-means centroids
the subject of my problem is related to cluster analysis. Specifically, with SPSS I am conducting a cross-validation procedure based on splitting the sample data into two independent halves (subsample ...
- 51
1
vote
0
answers
24
views
why does SVM outperforms KNN in 1-gram but in 2,3,4, and 5 KNN outperforms SVM?
my project is authorship attribution which is a multiclass classification, the number of classes is 150, and the number of documents is 2798; it is also an unbalanced issue some classes have more ...
- 11
1
vote
1
answer
324
views
Clustering by using Locality sensitive hashing *after* Random projection
It is well known that Random Projection (RP) is tightly linked to Locality Sensitive Hashing (LSH). My goal is to cluster a large number of points lying in a $d$-dimensional Euclidean space, where $d$ ...
2
votes
0
answers
217
views
kNN Classifier Asymptotic Error Rate versus Bayes Error Rate
Suppose we are in the realm of $M$ class classification, $M \in \mathbb{N}$. I have seen the following result stated many times, but only proven for the case $k = 1$. I would like to prove it for ...
- 163
2
votes
1
answer
187
views
Example of KNN overfitting with k=1
I know that with k=1 a KNN lead to overfitting, this is because it follows the noisy data of the training sample and not generalize well on new input sample. But I am confused on how this happens, I ...
1
vote
0
answers
45
views
What is the best way (in terms of performance and accuracy) to perform value estimation/prediction?
How do I predict the value of an item given its features and attributes? What is the best approach to this regression problem, in terms of performance as well as accuracy? And how do I process this ...
- 111
2
votes
2
answers
601
views
KNN: Should we randomly pick "folds" in RandomizedSearchCV?
TL;DR
In KNN, K is the hyperparameter so we randomly pick it while performing RandomizedSearchCV. Should we also randomly pick the split [Cross-validation + Train] after k-folding? I am considering ...
- 123
2
votes
1
answer
78
views
What does it mean having 1 as best k parameter in K-NN?
I'm working with a large dataset (761 rows and about 57k-60k features) and after doing a feature selection to select the best 10 features I'm using different ML algorithms to classify some cases. In ...
- 23
2
votes
1
answer
282
views
What's the consquences of training a KNN model over the whole data?
I know it is a basic question, but is the only consequence not having a good evaluation of the data?
is the accuracy or precision of the model affected in any way if there is no test set? worse ...
- 443
2
votes
2
answers
729
views
Higher accuracy in training set than test set in KNN classifier
If I got an accuracy of 95% in the training set, and 80% accuracy in the test set, what could be the explanation for that? it sounds like a pretty basic question, but I just can't put it into words ...
- 443
0
votes
1
answer
264
views
Why predicted values differ in knn regression when using caret vs FNN?
I was trying to do some manual calculations of knn regression and came across this unusual error. The predicted values done by hand do not match with the ones I got from the 'knnreg' function in the '...
- 41
1
vote
0
answers
178
views
Feature scaling with categorical variables
I've a dataset with numeric features and categorical features. For the latter I created dummy variables.
I've to implement a KNN model so I've to scale my variables. My doubt is: how to handle these ...
- 41
2
votes
1
answer
354
views
KNN(k-nearest neighbor) algorithm as supervised and as unsupervised algorithm. What are the main differences? How can it be both?
On internet and in articles KNN ist mostly described as supervised algorithm. But recently I have find also few articles where it is mentioned as unsupervised algorithm.I cannot find articles that are ...
- 21
2
votes
0
answers
116
views
Transition matrix as a feature to feed machine learning algorithm
Currently, I am trying to replicate a paper to extend the research on the paper they create a transition matrix similar to this one:
my question is : how can I feed this information into my KNN or ...
- 21
2
votes
1
answer
967
views
Can we regularize/penalize the KNN model?
While reading through KNN in detail, I was checking if there is a way to improve/penalize KNN?
I didn't find any concrete/easily understandable solutions.
- 21
2
votes
1
answer
77
views
How is a distance metric used in KNN when $K$ is given?
I am new to non-paramtric methods.
The conditional probability for classification using KNN is gen by:
$$
P(y=c|x,D)=\frac{1}{K}\sum_{n\in N_K(x,D)}I(y_n=c)
$$
where $N_K(x,D)$ is the set such that $K$...
- 605
0
votes
1
answer
37
views
How to infer nearest neighbors using distance metrics
My team and I are trying to identify group of customers to target for an investment promotion exercise. We decided to use the control group (which already are a part of this investment exercise) and ...
- 101
2
votes
0
answers
33
views
Any theory on whether good choices of $k$ depend on $N$ and $D$ in KNN classification?
I am well aware that cross validation is a usual method for selecting hyperparameters. However, I am looking for theoretical guidance on how to pick $k$, the number of neighbors, for a $k$-nearest-...
- 641
3
votes
0
answers
1k
views
How many features/dimensions/variables can K-Nearest Neighbors handle with a finite amount of data?
KNN is an algorithm where the "Curse of Dimensionality" applies extremely literally and directly.
Let's take some kind of basic, 50/50 balanced, binary classification problem. I'm wondering ...
- 1,531
2
votes
0
answers
570
views
Lower RMSE but worse model prediction
I am using a KNN model to predict quantity sold for a highly seasonal business. I chose KNN because I thought that using nearest neighbors would inform my model about said seasonality better than a ...
- 21
1
vote
0
answers
54
views
Bootstrapping with 1-NN classifier
Given the naive bootstrap estimate of the error of a classifier defined as:
$$\frac{1}{N}\frac{1}{B}\sum_{b=1}^{B}\sum_{i=1}^{N} L(y_i, \hat{c}_b(x_i))$$
Where $L(y, c(x)) = \mathbb{1}\{c(x)=y\}$ is ...
- 123
1
vote
0
answers
408
views
K-fold cross validation for kNN Imputer in Python [closed]
I have a dataset with columns, say, y, x1, x2, x2 and a lot of missing values in x1, x2, x3. I decided to use ...
- 350
1
vote
0
answers
187
views
Why does a Decision Tree algorithm outperform the Random Forest Algorithm in certain cases?
Currently I write my master thesis that deals with the binary prediction of university dropouts (dropout - yes/no). In the thesis, I compare the performance of three different classification ...
- 21
1
vote
1
answer
985
views
Is there a way to locate automatically the knee of the k-nearest neighbour graph in a DBSCAN analysis? [duplicate]
I am trying to write a function in R that automatically chooses the optimal parameters epsilon and MinPts in a DBSCAN analysis. I found that the k-nearest neighbour plot was very useful in order to ...
- 11