k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

learn more… | top users | synonyms

1
vote
0answers
9 views

Python kNN vs. radius nearest neighbor regression

Python offers two nearest neighbor regressions: radius nearest neighbor and k-nearest neighbor. I'm trying to figure out a few things: 1. Under which circumstances would each be preferable? 2. How do ...
0
votes
0answers
22 views

How can I use the output of KODAMA to predict unknown data points?

I can use KODAMA to create a model that classifies input data into two groups by setting the W vector to indicate the group and fix to a vector of all ...
1
vote
0answers
15 views

Does centering or mean normalizaiton alone every help in feature scaling?

In feature scaling, one way is to subtract the mean (centering) and then divide by the standard deviation for all data points. Suppose we just centered the data and didn't divide by the standard ...
0
votes
0answers
7 views

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor?

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor when no test set is available, --when the input vectors are unknowns? Here's my problem, I have a massive ...
2
votes
1answer
56 views

How to handle data normalization in kNN when new test data is received

I had a discussion with my colleagues about the following problem: Lets say we have 100 points of labeled data and we are using $k$-nearest neighbor method for prediction. So our data looks like ...
6
votes
1answer
53 views

VC-Dimension of k-nearest neighbor

What is the VC-Dimension of the k-nearest neighbor algorithm if k is equal to the number of training points used? Context: This question was asked in a course I take and the answer given there was ...
0
votes
0answers
17 views

data treatment before or after train/test sets split?

I have a variable in my data with NA values and I want to apply knn input. Should I do it before or after split the data in train and test set? If I do it after, each set will only use the values in ...
3
votes
1answer
67 views

What does the k-value stand for in a KNN model?

What is the k-value in a KNN classification model? Is K the number of Clusters?
1
vote
0answers
18 views

KNN converging to regression function proof (ESL page 19)

In Elements of Statistical Learning, page 19, it says I am confused what it means by "under mild regularity condition" and how one can prove the statement $\hat{f}(x) \rightarrow E(Y|X = x)$ under ...
0
votes
0answers
12 views

Smoothing of a 2D Empirical Distribution

I have a number of data points $\theta \in \mathbb{R}^2$ with corresponding values $x \in \mathbb{N}$. I am assuming the $x$ are realisations from a distribution $f(X | \theta)$. Given I have a lot ...
1
vote
1answer
148 views

If Manhattan distance always performs better on a dataset…what does it mean?

I'm analyzing my dataset using kNN. I experimented with various distance functions but Manhattan seems to perform better in terms of lowest RMSE over various values of k. I've read a bit about ...
5
votes
1answer
68 views

Explanation of formula for median closest point to origin of N samples from unit ball

In Elements of Statistical Learning, a problem is introduced to highlight issues with k-nn in high dimensional spaces. There are $N$ data points that are uniformly distributed in a $p$-dimensional ...
1
vote
0answers
15 views

Can KNN theoretically be less accurate than pattern averaging?

My task is pattern recognition. I need to classify 2D matrices into an arbitrary number of classes. The question is: For pattern classification, could k nearest neighbours algorithm ever be less ...
0
votes
0answers
14 views

What is the cost of finding neighbors in eps radius?

Finding neighbors in eps radius a sample point is called region query. There are data structures reduce cost of such queries. These structures are also used to give k-nearest neighbors of given sample ...
0
votes
0answers
85 views

Naive Bayes Nearest Neighbor (NBNN) implementation problems in MATLAB

I'm currently trying to classify the CIFAR-10 image dataset. I cam across a number of papers praising the the results from a non-parametric approach called Naive Bayes Nearest Neighbors. It uses SIFT ...
2
votes
3answers
83 views

Choosing optimal K for KNN

I performed a 5-fold CV to select the optimal K for KNN. And it seems like the bigger K gets, the smaller the error... Sorry I didn't have a legend, but the different colors represent different ...
1
vote
0answers
81 views

Distance measure for categorical attributes for k-Nearest Neighbor

For my class project, I am working on the Kaggle competition - Don't get kicked The project is to classify test data as good/bad buy for cars. There are 34 features and the data is highly skewed. I ...
0
votes
0answers
9 views

Find input image (ID,passport) in imagesDB based on similarity

I would like to decide if image is exists on DB images (pictures of IDs,passport,Stu. card,etc) I thought of KNN alghorithem that will plot the K closest images. Options for distance metric: 1) sum ...
0
votes
1answer
51 views

Are there optimization methods for k-NN parameter $k$?

Currently I am just go through from the min to the max, and determine $k$ by the performance. I am wondering if there's optimization approaches for selecting $k$? I am aware of there's question in ...
2
votes
0answers
35 views

Finding nearest neighbors using Jaccard distance for positive, real-valued vectors

Say we have $x_i, \ldots, x_n \in R ^ D$ with positive, real components and use Jaccard distance $$d(x_i, x_j) = 1 - \frac{\sum_{d = 1}^D\min(x_i^d, x_j^d)}{\sum_{d = 1}^D\max(x_i^d, x_j^d)}$$ to ...
0
votes
1answer
87 views

How to solve this problem on Curse of Dimensionality problem - Nearest Neighbours

I have started learning classification techniques and trying to solve the problems from the book Introduction to Statistical Learning. While currently working on the which is based on Curse of ...
4
votes
3answers
311 views

When should I apply feature scaling for my data

I started a discussion with a collague of mine and we started to wonder, when should one apply feature normalization / scaling to the data? Lets say that we have a set of features with some of the ...
0
votes
0answers
64 views

Binary classification with KNN

I post here because I don't know how to improve the performance of my binary KNN. The problem is that I have 99.8% Specificity and only 82% Sensitivity, but I'd rather have more Sensitivity than ...
0
votes
0answers
41 views

KNN - does it use centroids?

I'm really confused now having read so many articles on KNN, I can't help but think I'm missing the obvious. Let's say I have persons P1 and P2, P3 are represented with attributes of height, weight ...
0
votes
4answers
102 views

Is kNN best for classification?

I wanted to know if kNN might produce the best result for classification? Since, it is not model based, it does not loose any detail and compares every training sample to give the prediction. Hence ...
0
votes
0answers
20 views

Why SVM with RBF is similar to KNN with prototypes search?

I explored similar questions and everything I can see is that both kNN and RBF are non-parametric methods to estimate the density of probability of your data. However, I am not sure if this has ...
0
votes
0answers
42 views

How to use Similarity Measure in K-nearest neighbor Classification?

I have a similarity measure just like cosine. How can i use that similarity measure in traditional k-nn classification? Please provide some literature review (research papers) details which i should ...
1
vote
1answer
40 views

Classification tips for a begginer

I'm doing a graduation work that involves applying Classification algorithms in a dataset of matches from Dota 2 (a popular MOBA game). Here's an explanation of the problem: Dota 2 matches are played ...
0
votes
0answers
14 views

Several categories using k-nearest neighbour

Is it possible to have a training data set for the k nearest neighbour with several categories likes 700 and about 10 attributes?
1
vote
2answers
174 views

Doing low-dimensional KNN on a large dataset

I have a simple two-dimensional dataset with columns X1,X2, and [outcome], and I want to try KNN (probably K around 100 or 1000, though ideally CV would be possible). The problem is that my dataset ...
0
votes
1answer
45 views

predict category by using K-NN algorithm having text features

I would like to predict the category of the provided data by using K-NN algorithm. Here is an example of the training data set ...
0
votes
1answer
39 views

Classifying a set of photos to places

I want to cluster photos and map them to places. As input I have Photos with locations (lat, long) Places - some as (imprecise) bounding boxes, some just as points, maybe others as bounding ...
0
votes
0answers
37 views

Method to select meaningful features for nearest neighbor classification

i try to perform some k nearest neighbor classification in R. That for i want to select the most meaningful features to deal with the curse of dimensionality. I have already decided to use Mahalanobis ...
2
votes
3answers
391 views

Does k-NN with k=1 always implies overfitting?

I found somewhere such statement, but on the other hand in some sources I found, that it is ok. What about risk of overfitting while using 1-NN in binary classification problem where explanatory ...
0
votes
1answer
161 views

advantage of euclidean distance for classification

Has euclidean distance any advantage in compare to another distance based methods like Manhatan distance or Maximum difference metric?
6
votes
2answers
316 views

Is KNN a discriminative learning algorithm?

It seems that KNN is a discriminative learning algorithm but I can't seem to find any online sources confirming this. Is KNN a discriminative learning algorithm?
1
vote
2answers
224 views

How to deal with unbalanced data

I'm doing data analysis with a dataset of 11795 data points (with 88 features). 85% (9973 points) of these data points correspond to data points belonging to class 1, 5% (589 points) belong to class 2 ...
0
votes
0answers
24 views

sample size to estimate the best k in kNN

I would like to use a kNN classifier. My data set is quite small and include 2 classes having about 200 samples each one. I need to estimate k by using a cross-validation approach. How many samples ...
10
votes
3answers
1k views

Why would anyone use KNN for regression?

From what I understand, we can only build a regression function that lies within the interval of the training data. For example (only one of the panels is necessary): How would I predict into the ...
2
votes
1answer
496 views

Pros of Jeffries Matusita distance

According to some paper I am reading, Jeffries and Matusita distance is commonly used. But I couldn't find much information on it except for the formula below ...
0
votes
2answers
64 views

Regression: what to do with categorical predictors?

I've got a few categorical predictors (like gender,...) and now I want to build regression models. So I've made the categorical predictors numeric by for example: "female" --> 1 and "male" --> 0. But ...
0
votes
1answer
57 views

Is this a correct interpretation of k nearest neighbours?

Given this dataset : name1,name2,distance a,b,1 a,c,5 b,c,8 If k=1 is the following correct : ...
2
votes
1answer
164 views

kozachenko-leonenko entropy estimation

I'm trying to implement the entropy estimation based on the closest neighbour from Kozachenko and Leonenko but I'm facing a problem I can't solve. The idea is to work in a new set ...
0
votes
2answers
61 views

Is this a valid use case for Euclidean distance?

I have a set of points which is a count of links that users have clicked on : ...
0
votes
1answer
30 views

Looking for measurement for nonlinear model

I use KNN regression to train out a model. The model estimates running time of a program based on different inputs, and the output is a single variable, which is time (double type). I want to ...
0
votes
0answers
39 views

Best way to classify a set through a single feature

I need to classify a single dataset through a numeric value. I added below a simple dataset to explain what I need. Restriction: Category has two values: 1 or ...
1
vote
0answers
57 views

What criteria are used to compare feature-based classification techniques?

When comparing feature-based classification techniques, what characteristics about the different processes should be considered? I'm comparing different classification techniques to try to figure ...
0
votes
0answers
142 views

Feature Selection for look alike modeling using k-NN

I have a list of items and various parameters for each items. For each item on my list i need to identify items which are similar to the item from my whole population . I am planning on using K-NN ...
0
votes
1answer
74 views

All nearest neighbors in high dimensional space

Suppose I have a very large binary matrix representing $n$ customers and the $m$ products they bought, with $n$ and $m$ both rather large (in the order of millions). The matrix is also pretty sparse. ...
2
votes
1answer
953 views

How to define the maximum k of the kNN classifier?

I am trying to use kNN classifier to perform some supervised learning. In order to find the best number of 'k' of kNN, I used cross validation. For example, the following codes load some Matlab ...