Questions tagged [k-nearest-neighbour]

k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

Filter by
Sorted by
Tagged with
0
votes
1answer
23 views

How does knn regression .predict() work?

For a typical regression algorithm like linear regression, the model is y=2x+1 for instance. We can make predictions y = 3 when x=1 Picture above is an example from github.The green ...
9
votes
1answer
187 views

The No-Free-Lunch Theorem and K-NN consistency

In computational learning, The NFL theorem states that there is no universal learner. For every learning algorithm , there is a distribution that causes the learner output a hypotesis with a large ...
0
votes
1answer
21 views

What does $w_{ni}$ mean in the weighted nearest neighbour classifier?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input ...
0
votes
1answer
15 views

Is it necessary to scale the dependent variable in k-NN regression?

I want run kNN analysis to predict Y (continuous variable). I know that it is necessary to normalize all of the Xs. My question: is it also necessary to normalize Y values?
0
votes
0answers
6 views

improving performance of KNN classifier on FFT

I need to classify whether a product is passing or failing based on a noise check. I have 100 labeled "good" product and 100 "bad" product. For each, I recorded the sound for 3 seconds and each chunk ...
1
vote
2answers
33 views

How Rapidminer handle same distance for KNN Algorithm

Actually I already asked in rapidminer forum, but no one has given an answer yet.. https://community.rapidminer.com/discussion/55963/how-k-nn-algorithms-work-with-same-distance-in-rapidminer#latest ...
0
votes
0answers
24 views

Extension of the K-Nearest neighbor algorithm to get results in different neighborhoods

I would like to use the kNN algorithm to find the closest neighbor to a vector. But I would like it to limit to a point per neighborhood (radius) In this image, given the point in red, I would like ...
0
votes
2answers
21 views

Using KNN for audio classification based on FFT

I need to classify whether a product sound "good" and "bad" based on FFT of its audio recording. The FFT magnitudes are show for frequencies from 0 to 7khz, with a frequency resolution of 5 hz, so ...
2
votes
0answers
24 views

Minimize element-wise distance between two sets of points in R^n

Given two ordered sets $X, Y$ each containing $m$ elements in $\mathbb{R}^n$, I'm looking for a permutation $\sigma$ of the second set that minimizes $$\sum_{i=1}^m \lVert X_i - Y_{\sigma(i)} \rVert$$...
0
votes
0answers
3 views

How to determine the discount percentage of a product for a given product category and brand?

We are performing the analysis of data of an online shopping site. Please refer to the dataset mentioned in this link The fields of the dataset are: We have been asked to do the following: Perform ...
0
votes
0answers
19 views

Can I use KNN imputation when data are not MAR (Missing at Random)

I have a dataset with 891 observations and 12 variables. From them, 2 have NA values (V1 has 20% NA and V2 has 77% NA). First I examined if the data are MAR. So, I have created 2 new variables named ...
0
votes
0answers
16 views

Learning distance metric from output of knn

Consider a set $\mathcal X$ of points $\{x_1,\dots,x_n,x_{n+1},\dots,x_{n+m} \}\subset \mathbb R^p$. Let $A$ be some $p\times p$ matrix, unknown to you. Consider the set $$\mathcal X_A:=\{y_1,\dots,y_{...
0
votes
1answer
50 views

Is Hierarchical clustering a special case of knn(specific n=1)?

I'am working on time series in the scope of similarity detection at the moment. What seems to be a well researched approach is dynamic time warping in combination with k-1NN as classification ...
0
votes
0answers
29 views

Is it possible to perform kNN imputation for missing values in time-series data?

I have energy consumption data containing the time values and and corresponding consumption values in kWh. I want to perform imputation in R.
0
votes
0answers
8 views

Reduce scoring time for test sample of K-Nearest Neighbors regression for a time series dependent variables

My dependent variable is values of $ balance amount for an entity over next 24 months after the entity is subjected to a specific treatment. I am trying to predict these 24 values (bal1-bal24) for ...
0
votes
1answer
37 views

What is the VC dimension of k-nearest-neighbours with k=1?

I would answer that it is $\infty$, but I have a gut feeling this may not be the correct answer... May I present my proof attempt that it is indeed $\infty$, so that you can clear any misconceptions ...
0
votes
0answers
16 views

Time complexity for locality sensitive hashing similar image search

I am trying to find most visually similar images for large image dataset. (N=1 million), using LSH (Locality Sensitive Hashing). Image feature vectors are 4096 dimensional VGG-16 features. Now, my ...
0
votes
1answer
49 views

R prcomp and KNN different correct classification rate

I'm performing a classification task using KNN and PCA to pre-process the data. The dataset contains 101 continuous variables and the column of the labels (here the link to download the data filebin....
1
vote
1answer
30 views

How does caret resolve ties in the KNN classification? [closed]

I have a multi-class classification problem, in which I'm using caret package k nearest neighbour classifier, (4 classes), which means that an odd number for k won't prevent classification ties. So ...
2
votes
1answer
29 views

k-means/k-nearest neighbours on multi-dimensional scaled data

I used the Python manifold library for multi-dimensional scaling on my distance matrix. Can I use k-means or k-nearest neighbours on ...
1
vote
0answers
26 views

How to select the most important features, categorical & numerical data

I need to find out which factors are relevant when predicting low birth weight. My model looks like this: ...
0
votes
0answers
9 views

Calcule in-knn or out-knn (in-degree of nearest neighbours) of network. Assortativity in- out-degree and in- out-knn

I am trying to calcule using igraph the in-degree of nearest neighbours (in-knn) of a network or out-degree of nearest neighbours (out-knn). How can I do it in igraph (R)? This question is to ...
0
votes
1answer
34 views

How valid is this Stacking Model (input features to weak learners are different)?

I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type. I have to predict the labels ( 10 in number) for them. I tried applying several models and came to a ...
1
vote
1answer
32 views

What units is my mean squared error if I center and scale my training data?

I have a KNN model that I used to predict the close price on houses. ...
1
vote
1answer
42 views

How do I avoid time leakage in my KNN model?

I am building a KNN model to predict housing prices. I'll go through my data and my model and then my problem. Data - ...
0
votes
0answers
35 views

What does the intercept represent in a model matrix?

I am making a KNN algorithm to predict close_price with about 80,000 rows of this data. ...
0
votes
0answers
16 views

Different k’s for KNN

I would like to see the knn model performance on my data for various values of k. Can I just take the same training data and compute on it the knn for different k values or should I do cross ...
1
vote
2answers
222 views

Determining epsilon for DBSCAN

I'm using the method described in this paper for determining the optimal epsilon value for DBSCAN clustering in which a plot of the nearest neighbors is used: However, the plots in the paper and ...
0
votes
0answers
37 views

How to choose the right forecast method for variable 'X' when I have some available forecast for variable 'Y' with historical data of X and Y?

I have yearly historical data for variables 'X' and 'Y'. Say the time frame is 't'. In addition to available historical data, I also have the forecast data of variable 'Y' for t+1. My aim is to ...
0
votes
0answers
17 views

Query complexity of ANN based on Hamming distance

Ilya R provides the following query complexity of ANN for Hamming distance based on coordinate sampling https://www.ilyaraz.org/static/class_2018/files/20181023.pdf When he says ...
0
votes
0answers
24 views

Knn Classification on iris dataset

I'm following along https://rpubs.com/Drmadhu/IRISclassification to understand Knn classification. Here's the code I have: library(FNN) iris.sample<-sample.int(n=nrow(irisdat),size=floor(0.75*...
1
vote
1answer
38 views

Construction of confusion matrix when cross-validating with k-NN in R

I've a dataset looking like this: ...
0
votes
1answer
100 views

Estimation of average treatment effect based on nearest neighbor matching [closed]

I would like to use R to duplicate the treatment effect estimation method used in Stata. Specifically, this is the Stata method I would like to duplicate. I have tried the package ...
1
vote
0answers
13 views

Why is “condensed nearest neighbour” Parametric? [duplicate]

Definition of "condensed nearest neighbour", at training time it chooses the c "best" training examples (where c is a hyper-parameter), and at test time uses the usual KNN prediction but based only on ...
1
vote
0answers
18 views

Why is “consistent nearest neighbour” Non-parametric? [duplicate]

Definition of "Consistent nearest neighbour", runs our usual KNN classifier but instead of viewing k as a hyper-parameter it always sets k = ceil[log(n)]. So far, I looked-up many references and ...
0
votes
0answers
20 views

Data leakage in multilevel validation

I participate in competition that have historical data. I break it down according to this scheme. ...
0
votes
0answers
36 views

Optical Character Recognition - digits on the screen

My task is to classify a digit based on a small image containing one digit only. The font type and size is the same across the training/test dataset, but the position of the digit in the image might ...
1
vote
0answers
17 views

k-Nearest Neighbors multiclass - what if we can't form the majority?

I'm watching a lecture on applying kNN to the digit recognition (MNIST dataset) problem. Let's say $n$ denotes the number of neighbors. This is a multiclass classification problem, so having an odd ...
0
votes
0answers
31 views

How does model accuracy compare across the folds in cross validation

If we have two cross validation from KNN and linear regression like this: KNN: ...
1
vote
1answer
106 views

Loss function for KNN Regressor

What is the Loss function for KNN Regressor? Would it be similar to OLS? If so what would be the main difference?
0
votes
1answer
35 views

Predict Click Through Rates for Google depending on the Position

I'm given the task to calculate expected click through rates (CTR) for rankings of a given site in Google, using the data Google provides in their Google Search Console. What I get there is basically ...
1
vote
2answers
28 views

Hypothetically, can we perfectly classify new instances given infinite support from validation and training? KNN

Hypothetically speaking, Given infinite amounts of training data and validation data, can we achieve perfect classification (score of 1) given ML algorithms such as KNN? Thank you.
1
vote
1answer
55 views

Dimensionality Reduction for Optimally Preserving KNN

Do any dimensionality reduction techniques find embeddings which optimally preserve the K-nearest neighbors of each point? If no algorithm provably does this, are there algorithms which heuristically ...
0
votes
1answer
25 views

Training data grow with K-NN and Naive Bayes

My doubt is about the grow of the training data using K-NNs and Naive Bayes. As it grows larger, does prediction (on test data) become also computationally harder?
0
votes
0answers
53 views

Is this an example of overfitting?

I am trying to predict some future values using either KNN or regression model. I have about 9 independent variables that do not seem to have strong correlation to each other (Not completely sure ...
0
votes
1answer
31 views

A simple concept question regarding Bootstrapping

I am having a difficulty understanding whether I can use the bootstrapping method for prediction. First off, my data is as follows. where personal Income is the dependent variable (y), and GPA, Age, ...
0
votes
0answers
21 views

k nearest neighbor for likert data

I have a data set which contains several variables. All these variables are catgorical variables. I want to use k nearst neighbour method. When I am using this technique do I need to convert the data ...
0
votes
0answers
31 views

How to evaluate the goodness of a KNN-based recommender system?

I'm building a content-based recommender system and I'm using KNN. That means, for each test instance I'm using KNN to find the most similar objects, and then come up with a recommendation. However I'...
1
vote
1answer
82 views

Propensity score matching: bias adjustment

I'm using propensity score matching to match similar individuals. I.e., I first estimate a propensity score (the probability of treatment conditional on some set of variables) and then match on the ...
3
votes
1answer
38 views

Quantifying a manifold folding unto itself

I have a dataset of ~7k scattered points in 3D which represents a manifold that may or may not "fold unto itself". Here's an example where this does happen (look at the top-right yellow triangles): ...