Questions tagged [k-nearest-neighbour]

k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

Filter by
Sorted by
Tagged with
0
votes
0answers
23 views

How to run a jupyter notebook code on multiple cores? [closed]

I am trying to implement a K-Neighbours Classification model on a dataset with shape (60000,32,32) on my system (16 GB ram, I5 8th gen processor, 256 GB hard disk). Though I have normalized the data, ...
0
votes
0answers
12 views

Is kNN viable when predictors are normally distributed and the data is unbalanced?

I'm considering a 3-dimensional ($X_{1}, X_{2}, X_{3}$) classification problem where $X_{i}\sim N(0,1)$; observations are to be classified into one of 8 categories ($2^{3}$), reflecting (High, Low) ...
0
votes
0answers
38 views

What is the relation between MSE of K-NN for a regression problem and LOOCV?

I trying to answer this question: Denote the MSE of K-NN for a regression problem: $𝐸_{𝑖𝑛} =\frac{1}{𝑛}\sum_{i=1}^n(𝑦_𝑖 − \frac{1}{𝑘}\sum_{j=1}^k𝑦_𝑗)^2$ , where for each $𝑦_𝑖$, the ...
0
votes
1answer
12 views

What's the formal term for the “Group of points that have X as their neighbor”?

Due to asymetrical nature of K-NN, the points neighboring X need not be the same as the points which have X as their neighbor. Is there a formal term to designate those points which have X as one of ...
1
vote
0answers
20 views

Hot Deck Imputation

I am currently facing a set of data with missing values. I would like to impute these values with a Hot Deck. I have read upon the Hot Deck methods and decided to use Nearest Neigbour Hot Deck (NNHD) ...
1
vote
0answers
19 views

KNN problem with outlier

May I ask why a KNN model with K = 1 will have a strange blue dot on the left hand side of the Bayes decision and KNN decision boundary? I extracted this picture from the ISLR
0
votes
0answers
6 views

Can there be an overlap in finding prospects based on current customers using the K-nearest algorithm?

Based on this paper: http://wps-feb.ugent.be/Papers/wp_13_863.pdf (page 8 3.1 phase 1) The goal is to find prospects based on current customers. My first question is: How can they define this ...
0
votes
0answers
34 views

What is the purpose of the generalization error bound?

I could not understand what is the purpose of the generalization error bound, why do we need to calculate it?!. How does the generalization error bound work with 1-nearest neighbour algorithm ?. Does ...
0
votes
1answer
55 views

In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbours. Why not manhattan distance?

In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbours. Why not manhattan distance ?
1
vote
0answers
27 views

How to solve several issues with train fucntion in R?

I have to solve some problems with knn algorithm and corss validation. I will post my code and the text of the exercise later so you will be able to understand. At the end of the script R returns ...
2
votes
2answers
59 views

Does cross-validation apply to K-Nearest Neighbors given no estimated parameters?

Cross validation involves (1) taking your original set X, (2) removing some data (e.g. one observation in LOO) to produce a residual "training" set Z and a "holdout" set W, (3) fitting your model on Z,...
4
votes
1answer
76 views

What does '1-NN is statistically inconsistent' mean?

I am confused about the fact "1-NN is statistically inconsistent". Why and how?? References https://arxiv.org/pdf/1712.02369.pdf https://www.cs.bgu.ac.il/~karyeh/bayes-consistent-1nn.pdf https://...
2
votes
1answer
78 views

The expected error of 1 nearest Neighbor (1-NN) on large or infinite dataset

I have question regarding the expected error of 1NN. Assume the training set is large enough or infinite. let x' is a test point and r be its nearest point. the probability distribution of two classes(...
0
votes
0answers
27 views

Understanding the kdist graph used to select DBSCAN epsilon parameter

I need to use DBSCAN for my research and am having trouble understanding the kdist graph used to select the epsilon parameter - specifically, I do not understand what is happening behind the scenes ...
1
vote
2answers
49 views

Justification of kNN: sample mean vs expectation a binary variable?

Suppose, x is a random binary variable with values {0, 1}, and $E[x] > 0.5.$ Is it true that, for a random sample $S$ of $x$, $P[\mu_S(x) > 0.5] > 0.5.$ In other words, if expectation of ...
0
votes
0answers
18 views

Dimensionality reduction preserving K nearest neighbours

I am looking for a dimensionality reduction technique which preserves K nearest neighbours. My input is 800000 2400 dimensional count vectors from sklearn's CountVectorizer and I would like to find ...
0
votes
0answers
19 views

Need for dimensionality reduction in Breast Cancer Dataset

I was attempting to analyze the Wisconsin Breast Cancer Diagnostic dataset. Have a couple of questions / doubts. Per the attached paper, the performance metrics were worse after dimensionality ...
0
votes
0answers
38 views

Can knn be used for multivariate multiple regression?

I am working with MLB data with around 15000 observations for seasonal player stat. The data frame's structure looks like this (I'm making up the stats): ...
1
vote
0answers
35 views

Sparsity problem in KNN

There is something wrong with the following explanation regarding this figure: The data densities are 6.3%, 4.19%, 1.39% respectively, so that the degrees of sparsity are 93.7%, 95.81%, 98.61%. ...
3
votes
2answers
99 views

In the context of KNN, why small K generates complex models?

Section 1.4.8 of "Machine Learning: A Probabilistic Perspective by Kevin Patrick Murphy" gives this figure (Figure 1.21(a)) to illustrate the error rate of a KNN classifier for different values of k: ...
0
votes
1answer
155 views

What does the symbol for pi with a lower perpendicular mean?

What does mean in context of the equation
0
votes
1answer
58 views

How does knn regression .predict() work?

For a typical regression algorithm like linear regression, the model is y=2x+1 for instance. We can make predictions y = 3 when x=1 Picture above is an example from github.The green ...
10
votes
1answer
209 views

The No-Free-Lunch Theorem and K-NN consistency

In computational learning, The NFL theorem states that there is no universal learner. For every learning algorithm , there is a distribution that causes the learner output a hypotesis with a large ...
0
votes
1answer
38 views

What does $w_{ni}$ mean in the weighted nearest neighbour classifier?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input ...
0
votes
1answer
18 views

Is it necessary to scale the dependent variable in k-NN regression?

I want run kNN analysis to predict Y (continuous variable). I know that it is necessary to normalize all of the Xs. My question: is it also necessary to normalize Y values?
0
votes
0answers
7 views

improving performance of KNN classifier on FFT

I need to classify whether a product is passing or failing based on a noise check. I have 100 labeled "good" product and 100 "bad" product. For each, I recorded the sound for 3 seconds and each chunk ...
1
vote
2answers
124 views

How Rapidminer handle same distance for KNN Algorithm

Actually I already asked in rapidminer forum, but no one has given an answer yet.. https://community.rapidminer.com/discussion/55963/how-k-nn-algorithms-work-with-same-distance-in-rapidminer#latest ...
0
votes
0answers
38 views

Extension of the K-Nearest neighbor algorithm to get results in different neighborhoods

I would like to use the kNN algorithm to find the closest neighbor to a vector. But I would like it to limit to a point per neighborhood (radius) In this image, given the point in red, I would like ...
0
votes
2answers
57 views

Using KNN for audio classification based on FFT

I need to classify whether a product sound "good" and "bad" based on FFT of its audio recording. The FFT magnitudes are show for frequencies from 0 to 7khz, with a frequency resolution of 5 hz, so ...
2
votes
0answers
32 views

Minimize element-wise distance between two sets of points in R^n

Given two ordered sets $X, Y$ each containing $m$ elements in $\mathbb{R}^n$, I'm looking for a permutation $\sigma$ of the second set that minimizes $$\sum_{i=1}^m \lVert X_i - Y_{\sigma(i)} \rVert$$...
0
votes
0answers
3 views

How to determine the discount percentage of a product for a given product category and brand?

We are performing the analysis of data of an online shopping site. Please refer to the dataset mentioned in this link The fields of the dataset are: We have been asked to do the following: Perform ...
0
votes
0answers
46 views

Can I use KNN imputation when data are not MAR (Missing at Random)

I have a dataset with 891 observations and 12 variables. From them, 2 have NA values (V1 has 20% NA and V2 has 77% NA). First I examined if the data are MAR. So, I have created 2 new variables named ...
0
votes
0answers
22 views

Learning distance metric from output of knn

Consider a set $\mathcal X$ of points $\{x_1,\dots,x_n,x_{n+1},\dots,x_{n+m} \}\subset \mathbb R^p$. Let $A$ be some $p\times p$ matrix, unknown to you. Consider the set $$\mathcal X_A:=\{y_1,\dots,y_{...
0
votes
1answer
143 views

Is Hierarchical clustering a special case of knn(specific n=1)?

I'am working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
0
votes
0answers
99 views

Is it possible to perform kNN imputation for missing values in time-series data?

I have energy consumption data containing the time values and and corresponding consumption values in kWh. I want to perform imputation in R.
0
votes
0answers
9 views

Reduce scoring time for test sample of K-Nearest Neighbors regression for a time series dependent variables

My dependent variable is values of $ balance amount for an entity over next 24 months after the entity is subjected to a specific treatment. I am trying to predict these 24 values (bal1-bal24) for ...
0
votes
1answer
238 views

What is the VC dimension of k-nearest-neighbours with k=1?

I would answer that it is $\infty$, but I have a gut feeling this may not be the correct answer... May I present my proof attempt that it is indeed $\infty$, so that you can clear any misconceptions ...
0
votes
0answers
23 views

Time complexity for locality sensitive hashing similar image search

I am trying to find most visually similar images for large image dataset. (N=1 million), using LSH (Locality Sensitive Hashing). Image feature vectors are 4096 dimensional VGG-16 features. Now, my ...
0
votes
1answer
57 views

R prcomp and KNN different correct classification rate

I'm performing a classification task using KNN and PCA to pre-process the data. The dataset contains 101 continuous variables and the column of the labels (here the link to download the data filebin....
1
vote
1answer
72 views

How does caret resolve ties in the KNN classification? [closed]

I have a multi-class classification problem, in which I'm using caret package k nearest neighbour classifier, (4 classes), which means that an odd number for k won't prevent classification ties. So ...
2
votes
1answer
40 views

k-means/k-nearest neighbours on multi-dimensional scaled data

I used the Python manifold library for multi-dimensional scaling on my distance matrix. Can I use k-means or k-nearest neighbours on ...
1
vote
0answers
26 views

How to select the most important features, categorical & numerical data

I need to find out which factors are relevant when predicting low birth weight. My model looks like this: ...
0
votes
0answers
23 views

Calcule in-knn or out-knn (in-degree of nearest neighbours) of network. Assortativity in- out-degree and in- out-knn

I am trying to calcule using igraph the in-degree of nearest neighbours (in-knn) of a network or out-degree of nearest neighbours (out-knn). How can I do it in igraph (R)? This question is to ...
0
votes
1answer
38 views

How valid is this Stacking Model (input features to weak learners are different)?

I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type. I have to predict the labels ( 10 in number) for them. I tried applying several models and came to a ...
1
vote
1answer
44 views

What units is my mean squared error if I center and scale my training data?

I have a KNN model that I used to predict the close price on houses. ...
1
vote
1answer
59 views

How do I avoid time leakage in my KNN model?

I am building a KNN model to predict housing prices. I'll go through my data and my model and then my problem. Data - ...
0
votes
0answers
86 views

What does the intercept represent in a model matrix?

I am making a KNN algorithm to predict close_price with about 80,000 rows of this data. ...
0
votes
0answers
24 views

Different k’s for KNN

I would like to see the knn model performance on my data for various values of k. Can I just take the same training data and compute on it the knn for different k values or should I do cross ...
1
vote
2answers
890 views

Determining epsilon for DBSCAN

I'm using the method described in this paper for determining the optimal epsilon value for DBSCAN clustering in which a plot of the nearest neighbors is used: However, the plots in the paper and ...
0
votes
0answers
39 views

How to choose the right forecast method for variable 'X' when I have some available forecast for variable 'Y' with historical data of X and Y?

I have yearly historical data for variables 'X' and 'Y'. Say the time frame is 't'. In addition to available historical data, I also have the forecast data of variable 'Y' for t+1. My aim is to ...