# Questions tagged [k-nearest-neighbour]

k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

426 questions
Filter by
Sorted by
Tagged with
23 views

### How to run a jupyter notebook code on multiple cores? [closed]

I am trying to implement a K-Neighbours Classification model on a dataset with shape (60000,32,32) on my system (16 GB ram, I5 8th gen processor, 256 GB hard disk). Though I have normalized the data, ...
12 views

### Is kNN viable when predictors are normally distributed and the data is unbalanced?

I'm considering a 3-dimensional ($X_{1}, X_{2}, X_{3}$) classification problem where $X_{i}\sim N(0,1)$; observations are to be classified into one of 8 categories ($2^{3}$), reflecting (High, Low) ...
38 views

### What is the relation between MSE of K-NN for a regression problem and LOOCV?

I trying to answer this question: Denote the MSE of K-NN for a regression problem: $𝐸_{𝑖𝑛} =\frac{1}{𝑛}\sum_{i=1}^n(𝑦_𝑖 − \frac{1}{𝑘}\sum_{j=1}^k𝑦_𝑗)^2$ , where for each $𝑦_𝑖$, the ...
12 views

### What's the formal term for the “Group of points that have X as their neighbor”?

Due to asymetrical nature of K-NN, the points neighboring X need not be the same as the points which have X as their neighbor. Is there a formal term to designate those points which have X as one of ...
20 views

### Hot Deck Imputation

I am currently facing a set of data with missing values. I would like to impute these values with a Hot Deck. I have read upon the Hot Deck methods and decided to use Nearest Neigbour Hot Deck (NNHD) ...
19 views

### KNN problem with outlier

May I ask why a KNN model with K = 1 will have a strange blue dot on the left hand side of the Bayes decision and KNN decision boundary? I extracted this picture from the ISLR
6 views

### Can there be an overlap in finding prospects based on current customers using the K-nearest algorithm?

Based on this paper: http://wps-feb.ugent.be/Papers/wp_13_863.pdf (page 8 3.1 phase 1) The goal is to find prospects based on current customers. My first question is: How can they define this ...
34 views

### What is the purpose of the generalization error bound?

I could not understand what is the purpose of the generalization error bound, why do we need to calculate it?!. How does the generalization error bound work with 1-nearest neighbour algorithm ?. Does ...
55 views

### In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbours. Why not manhattan distance?

In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbours. Why not manhattan distance ?
27 views

### How to solve several issues with train fucntion in R?

I have to solve some problems with knn algorithm and corss validation. I will post my code and the text of the exercise later so you will be able to understand. At the end of the script R returns ...
59 views

### Does cross-validation apply to K-Nearest Neighbors given no estimated parameters?

Cross validation involves (1) taking your original set X, (2) removing some data (e.g. one observation in LOO) to produce a residual "training" set Z and a "holdout" set W, (3) fitting your model on Z,...
76 views

### What does '1-NN is statistically inconsistent' mean?

I am confused about the fact "1-NN is statistically inconsistent". Why and how?? References https://arxiv.org/pdf/1712.02369.pdf https://www.cs.bgu.ac.il/~karyeh/bayes-consistent-1nn.pdf https://...
78 views

### The expected error of 1 nearest Neighbor (1-NN) on large or infinite dataset

I have question regarding the expected error of 1NN. Assume the training set is large enough or infinite. let x' is a test point and r be its nearest point. the probability distribution of two classes(...
27 views

### Understanding the kdist graph used to select DBSCAN epsilon parameter

I need to use DBSCAN for my research and am having trouble understanding the kdist graph used to select the epsilon parameter - specifically, I do not understand what is happening behind the scenes ...
49 views

### Justification of kNN: sample mean vs expectation a binary variable?

Suppose, x is a random binary variable with values {0, 1}, and $E[x] > 0.5.$ Is it true that, for a random sample $S$ of $x$, $P[\mu_S(x) > 0.5] > 0.5.$ In other words, if expectation of ...
18 views

### Dimensionality reduction preserving K nearest neighbours

I am looking for a dimensionality reduction technique which preserves K nearest neighbours. My input is 800000 2400 dimensional count vectors from sklearn's CountVectorizer and I would like to find ...
19 views

### Need for dimensionality reduction in Breast Cancer Dataset

I was attempting to analyze the Wisconsin Breast Cancer Diagnostic dataset. Have a couple of questions / doubts. Per the attached paper, the performance metrics were worse after dimensionality ...
38 views

### Can knn be used for multivariate multiple regression?

I am working with MLB data with around 15000 observations for seasonal player stat. The data frame's structure looks like this (I'm making up the stats): ...
35 views

### Sparsity problem in KNN

There is something wrong with the following explanation regarding this figure: The data densities are 6.3%, 4.19%, 1.39% respectively, so that the degrees of sparsity are 93.7%, 95.81%, 98.61%. ...
99 views

### In the context of KNN, why small K generates complex models?

Section 1.4.8 of "Machine Learning: A Probabilistic Perspective by Kevin Patrick Murphy" gives this figure (Figure 1.21(a)) to illustrate the error rate of a KNN classifier for different values of k: ...
155 views

### What does the symbol for pi with a lower perpendicular mean?

What does mean in context of the equation
58 views

### How does knn regression .predict() work?

For a typical regression algorithm like linear regression, the model is y=2x+1 for instance. We can make predictions y = 3 when x=1 Picture above is an example from github.The green ...
209 views

### The No-Free-Lunch Theorem and K-NN consistency

In computational learning, The NFL theorem states that there is no universal learner. For every learning algorithm , there is a distribution that causes the learner output a hypotesis with a large ...
38 views

### What does $w_{ni}$ mean in the weighted nearest neighbour classifier?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input ...
18 views

### Is it necessary to scale the dependent variable in k-NN regression?

I want run kNN analysis to predict Y (continuous variable). I know that it is necessary to normalize all of the Xs. My question: is it also necessary to normalize Y values?
7 views

### improving performance of KNN classifier on FFT

I need to classify whether a product is passing or failing based on a noise check. I have 100 labeled "good" product and 100 "bad" product. For each, I recorded the sound for 3 seconds and each chunk ...
124 views

### How Rapidminer handle same distance for KNN Algorithm

Actually I already asked in rapidminer forum, but no one has given an answer yet.. https://community.rapidminer.com/discussion/55963/how-k-nn-algorithms-work-with-same-distance-in-rapidminer#latest ...
38 views

### Extension of the K-Nearest neighbor algorithm to get results in different neighborhoods

I would like to use the kNN algorithm to find the closest neighbor to a vector. But I would like it to limit to a point per neighborhood (radius) In this image, given the point in red, I would like ...
57 views

### Using KNN for audio classification based on FFT

I need to classify whether a product sound "good" and "bad" based on FFT of its audio recording. The FFT magnitudes are show for frequencies from 0 to 7khz, with a frequency resolution of 5 hz, so ...
32 views

### Minimize element-wise distance between two sets of points in R^n

Given two ordered sets $X, Y$ each containing $m$ elements in $\mathbb{R}^n$, I'm looking for a permutation $\sigma$ of the second set that minimizes $$\sum_{i=1}^m \lVert X_i - Y_{\sigma(i)} \rVert$$...
3 views

### How to determine the discount percentage of a product for a given product category and brand?

We are performing the analysis of data of an online shopping site. Please refer to the dataset mentioned in this link The fields of the dataset are: We have been asked to do the following: Perform ...
46 views

### Can I use KNN imputation when data are not MAR (Missing at Random)

I have a dataset with 891 observations and 12 variables. From them, 2 have NA values (V1 has 20% NA and V2 has 77% NA). First I examined if the data are MAR. So, I have created 2 new variables named ...
22 views

### Learning distance metric from output of knn

Consider a set $\mathcal X$ of points $\{x_1,\dots,x_n,x_{n+1},\dots,x_{n+m} \}\subset \mathbb R^p$. Let $A$ be some $p\times p$ matrix, unknown to you. Consider the set \mathcal X_A:=\{y_1,\dots,y_{...
143 views

### Is Hierarchical clustering a special case of knn(specific n=1)?

I'am working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
99 views

### Is it possible to perform kNN imputation for missing values in time-series data?

I have energy consumption data containing the time values and and corresponding consumption values in kWh. I want to perform imputation in R.
9 views

23 views

### Time complexity for locality sensitive hashing similar image search

I am trying to find most visually similar images for large image dataset. (N=1 million), using LSH (Locality Sensitive Hashing). Image feature vectors are 4096 dimensional VGG-16 features. Now, my ...
57 views

### R prcomp and KNN different correct classification rate

I'm performing a classification task using KNN and PCA to pre-process the data. The dataset contains 101 continuous variables and the column of the labels (here the link to download the data filebin....
72 views

### How does caret resolve ties in the KNN classification? [closed]

I have a multi-class classification problem, in which I'm using caret package k nearest neighbour classifier, (4 classes), which means that an odd number for k won't prevent classification ties. So ...
40 views

### k-means/k-nearest neighbours on multi-dimensional scaled data

I used the Python manifold library for multi-dimensional scaling on my distance matrix. Can I use k-means or k-nearest neighbours on ...
26 views

### How to select the most important features, categorical & numerical data

I need to find out which factors are relevant when predicting low birth weight. My model looks like this: ...
23 views

### Calcule in-knn or out-knn (in-degree of nearest neighbours) of network. Assortativity in- out-degree and in- out-knn

I am trying to calcule using igraph the in-degree of nearest neighbours (in-knn) of a network or out-degree of nearest neighbours (out-knn). How can I do it in igraph (R)? This question is to ...
38 views

### How valid is this Stacking Model (input features to weak learners are different)?

I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type. I have to predict the labels ( 10 in number) for them. I tried applying several models and came to a ...
44 views

### What units is my mean squared error if I center and scale my training data?

I have a KNN model that I used to predict the close price on houses. ...
59 views

### How do I avoid time leakage in my KNN model?

I am building a KNN model to predict housing prices. I'll go through my data and my model and then my problem. Data - ...
86 views

### What does the intercept represent in a model matrix?

I am making a KNN algorithm to predict close_price with about 80,000 rows of this data. ...
24 views

### Different k’s for KNN

I would like to see the knn model performance on my data for various values of k. Can I just take the same training data and compute on it the knn for different k values or should I do cross ...