# Questions tagged [k-nearest-neighbour]

A non-parametric method of classification and regression. The input consists of the $k$ closest training examples in the feature space. The output is either the mode of the neighbors (in classification) or their mean (in regression).

536 questions
Filter by
Sorted by
Tagged with
34 views

### Can we regularize/penalize the KNN model?

While reading through KNN in detail, I was checking if there is a way to improve/penalize KNN? I didn't find any concrete/easily understandable solutions.
26 views

### How is a distance metric used in KNN when $K$ is given?

I am new to non-paramtric methods. The conditional probability for classification using KNN is gen by: $$P(y=c|x,D)=\frac{1}{K}\sum_{n\in N_K(x,D)}I(y_n=c)$$ where $N_K(x,D)$ is the set such that $K$...
23 views

### How to infer nearest neighbors using distance metrics

My team and I are trying to identify group of customers to target for an investment promotion exercise. We decided to use the control group (which already are a part of this investment exercise) and ...
33 views

### Any theory on whether good choices of $k$ depend on $N$ and $D$ in KNN classification?

I am well aware that cross validation is a usual method for selecting hyperparameters. However, I am looking for theoretical guidance on how to pick $k$, the number of neighbors, for a $k$-nearest-...
45 views

### How many features/dimensions/variables can K-Nearest Neighbors handle with a finite amount of data?

KNN is an algorithm where the "Curse of Dimensionality" applies extremely literally and directly. Let's take some kind of basic, 50/50 balanced, binary classification problem. I'm wondering ...
63 views

### Lower RMSE but worse model prediction

I am using a KNN model to predict quantity sold for a highly seasonal business. I chose KNN because I thought that using nearest neighbors would inform my model about said seasonality better than a ...
1 vote
19 views

### Bootstrapping with 1-NN classifier

Given the naive bootstrap estimate of the error of a classifier defined as: $$\frac{1}{N}\frac{1}{B}\sum_{b=1}^{B}\sum_{i=1}^{N} L(y_i, \hat{c}_b(x_i))$$ Where $L(y, c(x)) = \mathbb{1}\{c(x)=y\}$ is ...
1 vote
93 views

### K-fold cross validation for kNN Imputer in Python [closed]

I have a dataset with columns, say, y, x1, x2, x2 and a lot of missing values in x1, x2, x3. I decided to use ...
47 views

### Why does a Decision Tree algorithm outperform the Random Forest Algorithm in certain cases?

Currently I write my master thesis that deals with the binary prediction of university dropouts (dropout - yes/no). In the thesis, I compare the performance of three different classification ...
10 views

### Predictors for Classification Models using caret in R

I have been reading several different resources on classification and I am finding conflicting information. Some literature says caret's train() function assumes all predictors are numeric and the ...
1 vote
179 views

### Is there a way to locate automatically the knee of the k-nearest neighbour graph in a DBSCAN analysis? [duplicate]

I am trying to write a function in R that automatically chooses the optimal parameters epsilon and MinPts in a DBSCAN analysis. I found that the k-nearest neighbour plot was very useful in order to ...
1 vote
46 views

### Decide which K for KNN-model based on plot

I've been working with a dataset containing handwritten numbers, and to classify what number it is I've used KNN. I've made a plot comparing validation with training misclassification rate for each K =...
44 views

### K nearest neighbor VS Kernel density estimation (Parzen window)

I appreciate it if anyone could explain to me the advantages and disadvantages of knn and parzen relative to each other.
166 views

### Cosine similarity seems to perform better with higher dimensions than Euclidean distance? Should this be the case?

I've generated 100 random vectors (data points) in n∈[1,...,50] dimensions. I then compared distances between each pair of vectors and calculated the mean value. I've done this for all dimensions ...
54 views

### How is UMAP a valid dimensionality reduction technique when it uses KNN, which suffers from the curse of dimensionality?

I have not a found a satisfactory, or really any answer, to the following problem that I cannot resolve myself. UMAP is touted as an excellent dimensionality reduction technique by constructing a high-...
22 views

### Benchmarking a KD Tree vs a VP Tree

Hey so I am currently trying to use both a vp tree and a kd tree on a audio dataset, where i have about 10 features and the dataset consists of 1000 samples. For the KD Tree I use the sklearn package ...
61 views

### LOF vs k-NN in data with varying density

When the dataset is comprised of regions of varying densities, which technique is more effective for outlier detection Localised Outlier Factor(LoF) or K-Nearest neighbors (KNN)
1 vote
44 views

### How to construct training set for anomaly detection?

I am using a K-Nearest-Neighbor calculation as part of an outlier detection method, and I'm trying to decide how to construct the training dataset on which to base my KNN calculation for subsequent ...
1 vote
26 views

### 2NN classifier with neighbours having different labels

In 2-Nearest-Neighbourhood classifier, given a input x, and 2 neighbours of x have different labels, what can I assign as label for x? (Assuming I am not weighting neighbours based on distance)?
59 views

### K-nearest-neighbor - relationship between K, parameters and complexity

I want to understand the relationship (if any) between parameters and hyperparameters in a k-nearest-neighbor (KNN) model and how they relate to complexity. Assume A model which should classify ...
109 views

### Imbalanced classification using K-nearest neighbors classifier

For an NLP classification task I need to train two different classifiers and I've chosen to use a RandomForest and KNeighbors both the scikit-learn implementations. My dataset is strongly imbalanced. ...
97 views

### Bayes error and nearest neighbor classification

Upon studying for my midterm using Pattern Classification by Richard O. Duda, David G. Stork, Peter E.Hart (2001), I stumbled upon the following exercise: Using the solutions manual written by David ...
103 views

### How to get probabilities from KNN with cross-fold validation using caret::train in R

I am confused about the output of train() with knn models in R. My code below uses the Caravan data set, which comes included in the ISLR2 library: ...
324 views

### Why does error rate of kNN increase when k approaches size of training set?

I've been experimenting with the effect that different values of k have on the generalisation error of kNN classifier, and I've gotten some unexpected results towards the end when k approaches the ...
16 views

### What is the underlying intuitions behind the boundary bias problem?

I am trying to understand why k-nearest neighbors estimations suffer from bias at the boundary point of support. As far as I know, this is also true for kernel and random forest methods. If possible, ...
42 views

### Does KNN fail if the test data have no epsilon close nearest neighbors to the training data?

If I have binary-classification data and a Euclidean metric, and I know the best number of nearest neighbors, then I draw circles on my training data based on my K-value which tell me which regions ...
9 views

### Approximating the pdf at a sampled point using k-nearest neighbours?

I'm reading a (more or less classical) paper on Nearest-neighbour approximations to Entropy. At some point in the paper, given $N$ samples $X_1,...X_N$ from a $p$-dimensional random variable $X$ with ...
59 views

582 views

### Mean Accuracy and Standard Error of the Accuracy for KNN Classification algorithm

The given below code snippet is from the assignment of online course IBM ML with Python. Here's the assignment. The used variable names :mean_acc and ...
41 views

### K-Fold Cross Validation for K'-NN when K' greater than number of samples in each fold

I'm trying to evaluate the predictive power of a K'-NN algorithm for regression. My idea is to use K-Fold Cross Validation, but the problem is: what happens if I have a K' higher than the number of ...
4 views

### Algorithm to sqeeze a location structure

I'm looking an algorithm to squeeze the structure of the location of some items. The item are sensors that have a fixed location x,y on a floormap.and I want to sort them in an array like to be able ...
87 views

### Given specificity and sensitivity, how do we get ROC value for KNN [R]

I trained a KNN model with k fold cross validation, and I got the following results. I know how specificity and sensitivity are calculated, but the ROC value I don't know. How is that estimated? For ...
77 views

### KNeighborsClassifier Score with a precomputed user defined distance matrix

I am trying to implement a KNN from SKLEARN using an user defined distance matrix. I want to know which n_neighbors is giving the minimum error for my dataset. So, to avoid long calculation for ...
62 views

### Speed up Nearest Neighbour classification in highly duplicated data

I have a classification problem on a data set with 3 features and 2,000,000 examples. I have decided that I want to use K-Nearest Neighbors. Is there a way I can leverage the fact that the data only ...
1 vote
21 views

### KNN Distance measures [duplicate]

Why in KNN Euclidean distance is preferred over Manhattan for low dimension dataset whereas for high dimension, manhattan is preferred?
235 views

### What values of k in knn classifiers lead to overfitting and underfitting?

I am working on one of the common practice datasets "Breast Cancer" to create a model based on kNN classifiers. I am trying to find the number of k which leads to overfitting or underfitting ...