k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

learn more… | top users | synonyms

1
vote
0answers
17 views

Suspicious Amount of Zeros in Confusion Matrix

I have a data set with about 45000 observations and three features. When I apply machine learning classification algorithms like naive Bayes, kNN and SVM I receive a lot of zeros in the resulting ...
3
votes
3answers
358 views

KNN: 1-nearest neighbor

My question is about the 1-nearest neighbor classifier and is about a statement made in the excellent book The Elements of Statistical Learning, by Hastie, Tibshirani and Friedman. The statement is ...
0
votes
0answers
41 views

Using variance in place of standard deviation for z-normalization

I'm implementing a 1-nearest neighbor (with dynamic time warping as the distance measure) classification algorithm on a severely constrained embedded platform with no FPU, so we're doing fixed point ...
4
votes
3answers
118 views

Is it feasible to use k-Nearest Neighbours to identify text language?

I have seen various language identification libraries that claim to use naive Bayes classifier for text language identification, like CLD2 and language detector, but not any library that uses other ...
1
vote
0answers
14 views

Finding multiple records of the same person when the name cannot be trusted

I have a list of 100 of events, and each event has a list of thousands of persons. Some persons went to multiple events, and I would like to select all the events a person went to. I could just look ...
1
vote
0answers
59 views

Support Vector Machines vs KNN

It was my understanding that in a separable case, SVMs produce the best separation possible and therefore will always produce the same or a better classification rate compared with say, 1NN, ...
2
votes
1answer
67 views

How does Scikit Learn resolve ties in the KNN classification?

I have a multi-class classification problem, in which I'm using Scikit Learn's k nearest neighbour classifier, (5 classes), which means that an odd number for k won't prevent classification ties. So ...
5
votes
2answers
69 views

How can increasing the dimension increase the variance without increasing the bias in kNN?

My question is about understanding Figure 2.8 in The Elements of Statistical Learning (2nd edition). The topic of the section is how increasing dimension influence the bias/variance. I can roughly ...
3
votes
1answer
87 views

some inference about k-NN algorithms for better understanding? [closed]

I ran into some facts make me confusing. for k-NN classifier: I) why classification accuracy is not better with large values of k. II) the decision boundary is not smoother with smaller ...
0
votes
1answer
57 views

Locally weighted regression VS kernel linear regression?

I am trying to make it clear the relationship of the listed three methods. According to my understanding kernel regression means : the weight vector W lies in the space spanned by training data. $$ ...
0
votes
1answer
93 views

Buiding Ensemble model

I'm new to ensemble model. Suppose I've KNN models like this - (in R) library(class) ...
3
votes
1answer
92 views

Two broad categories of dimensionality reduction

As a starter in dimensionality reduction (DR), I recently acquired the following understanding. There are two very broad categories of DR techniques: We can compute an analytic form of mapping from ...
0
votes
1answer
21 views

Asymmetric distance measure in k-NN classifier?

What is the problem with an asymmetric distance measure in k-NN classifier? I think it will not cause problem, so long as I compute the distance consistently, say always from ...
3
votes
1answer
55 views

Cumulative match score

I have seen loads of graphs in papers of cumulative match scoring, but I can't find any information about what it means, or how it is created. A context that would be useful to see the explanation ...
0
votes
1answer
31 views

time series based classification

I want to classify some data. Basically the data is time series in nature. The target variable is categorical. I know there are so many algorithms for predicting the time series model. However, I have ...
0
votes
0answers
24 views

For time series classification, how can k nearest neighbors outperform other models?

Suppose you have a collections of time series data $Y^1,\ldots,Y^N$, $Y^i = Y^i_1,\ldots,Y^i_T$. Your training data consists of labels for some of these $Y^i$, and you wish to infer labels for the ...
0
votes
0answers
15 views

What does choice of metric say about the data?

I am using KNN method. I have three choices of metric (Euclidean, Manhattan and Chebyshev). The Error rate was minimum when I used Chebyshev distance. What does it say about the distribution data?
1
vote
0answers
39 views

Bias and variance of a naive bayes classifier and KNN classifier

After reading the paper by J. Friedman, ”On bias, variance, 0/1-loss, and the curse-of-dimensionality,” Data Mining and Knowl- edge Discovery, 1997. I would like to estimate both bias and variance ...
1
vote
0answers
54 views

Python kNN vs. radius nearest neighbor regression

Python offers two nearest neighbor regressions: radius nearest neighbor and k-nearest neighbor. I'm trying to figure out a few things: 1. Under which circumstances would each be preferable? 2. How do ...
0
votes
0answers
32 views

How can I use the output of KODAMA to predict unknown data points?

I can use KODAMA to create a model that classifies input data into two groups by setting the W vector to indicate the group and fix to a vector of all ...
1
vote
0answers
23 views

Does centering or mean normalizaiton alone every help in feature scaling?

In feature scaling, one way is to subtract the mean (centering) and then divide by the standard deviation for all data points. Suppose we just centered the data and didn't divide by the standard ...
0
votes
0answers
11 views

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor?

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor when no test set is available, --when the input vectors are unknowns? Here's my problem, I have a massive ...
2
votes
1answer
118 views

How to handle data normalization in kNN when new test data is received

I had a discussion with my colleagues about the following problem: Lets say we have 100 points of labeled data and we are using $k$-nearest neighbor method for prediction. So our data looks like ...
6
votes
1answer
152 views

VC-Dimension of k-nearest neighbor

What is the VC-Dimension of the k-nearest neighbor algorithm if k is equal to the number of training points used? Context: This question was asked in a course I take and the answer given there was ...
0
votes
0answers
29 views

data treatment before or after train/test sets split?

I have a variable in my data with NA values and I want to apply knn input. Should I do it before or after split the data in train and test set? If I do it after, each set will only use the values in ...
3
votes
1answer
102 views

What does the k-value stand for in a KNN model?

What is the k-value in a KNN classification model? Is K the number of Clusters?
1
vote
0answers
24 views

KNN converging to regression function proof (ESL page 19)

In Elements of Statistical Learning, page 19, it says I am confused what it means by "under mild regularity condition" and how one can prove the statement $\hat{f}(x) \rightarrow E(Y|X = x)$ under ...
0
votes
0answers
15 views

Smoothing of a 2D Empirical Distribution

I have a number of data points $\theta \in \mathbb{R}^2$ with corresponding values $x \in \mathbb{N}$. I am assuming the $x$ are realisations from a distribution $f(X | \theta)$. Given I have a lot ...
1
vote
1answer
170 views

If Manhattan distance always performs better on a dataset…what does it mean?

I'm analyzing my dataset using kNN. I experimented with various distance functions but Manhattan seems to perform better in terms of lowest RMSE over various values of k. I've read a bit about ...
5
votes
1answer
131 views

Explanation of formula for median closest point to origin of N samples from unit ball

In Elements of Statistical Learning, a problem is introduced to highlight issues with k-nn in high dimensional spaces. There are $N$ data points that are uniformly distributed in a $p$-dimensional ...
1
vote
0answers
16 views

Can KNN theoretically be less accurate than pattern averaging?

My task is pattern recognition. I need to classify 2D matrices into an arbitrary number of classes. The question is: For pattern classification, could k nearest neighbours algorithm ever be less ...
0
votes
0answers
15 views

What is the cost of finding neighbors in eps radius?

Finding neighbors in eps radius a sample point is called region query. There are data structures reduce cost of such queries. These structures are also used to give k-nearest neighbors of given sample ...
0
votes
0answers
146 views

Naive Bayes Nearest Neighbor (NBNN) implementation problems in MATLAB

I'm currently trying to classify the CIFAR-10 image dataset. I cam across a number of papers praising the the results from a non-parametric approach called Naive Bayes Nearest Neighbors. It uses SIFT ...
3
votes
3answers
156 views

Choosing optimal K for KNN

I performed a 5-fold CV to select the optimal K for KNN. And it seems like the bigger K gets, the smaller the error... Sorry I didn't have a legend, but the different colors represent different ...
1
vote
0answers
124 views

Distance measure for categorical attributes for k-Nearest Neighbor

For my class project, I am working on the Kaggle competition - Don't get kicked The project is to classify test data as good/bad buy for cars. There are 34 features and the data is highly skewed. I ...
0
votes
0answers
11 views

Find input image (ID,passport) in imagesDB based on similarity

I would like to decide if image is exists on DB images (pictures of IDs,passport,Stu. card,etc) I thought of KNN alghorithem that will plot the K closest images. Options for distance metric: 1) sum ...
0
votes
1answer
58 views

Are there optimization methods for k-NN parameter $k$?

Currently I am just go through from the min to the max, and determine $k$ by the performance. I am wondering if there's optimization approaches for selecting $k$? I am aware of there's question in ...
2
votes
0answers
43 views

Finding nearest neighbors using Jaccard distance for positive, real-valued vectors

Say we have $x_i, \ldots, x_n \in R ^ D$ with positive, real components and use Jaccard distance $$d(x_i, x_j) = 1 - \frac{\sum_{d = 1}^D\min(x_i^d, x_j^d)}{\sum_{d = 1}^D\max(x_i^d, x_j^d)}$$ to ...
0
votes
1answer
136 views

How to solve this problem on Curse of Dimensionality problem - Nearest Neighbours

I have started learning classification techniques and trying to solve the problems from the book Introduction to Statistical Learning. While currently working on the which is based on Curse of ...
4
votes
3answers
498 views

When should I apply feature scaling for my data

I started a discussion with a collague of mine and we started to wonder, when should one apply feature normalization / scaling to the data? Lets say that we have a set of features with some of the ...
0
votes
0answers
84 views

Binary classification with KNN

I post here because I don't know how to improve the performance of my binary KNN. The problem is that I have 99.8% Specificity and only 82% Sensitivity, but I'd rather have more Sensitivity than ...
0
votes
0answers
56 views

KNN - does it use centroids?

I'm really confused now having read so many articles on KNN, I can't help but think I'm missing the obvious. Let's say I have persons P1 and P2, P3 are represented with attributes of height, weight ...
0
votes
4answers
161 views

Is kNN best for classification?

I wanted to know if kNN might produce the best result for classification? Since, it is not model based, it does not loose any detail and compares every training sample to give the prediction. Hence ...
0
votes
0answers
60 views

Why SVM with RBF is similar to KNN with prototypes search?

I explored similar questions and everything I can see is that both kNN and RBF are non-parametric methods to estimate the density of probability of your data. However, I am not sure if this has ...
0
votes
0answers
66 views

How to use Similarity Measure in K-nearest neighbor Classification?

I have a similarity measure just like cosine. How can i use that similarity measure in traditional k-nn classification? Please provide some literature review (research papers) details which i should ...
1
vote
1answer
43 views

Classification tips for a begginer

I'm doing a graduation work that involves applying Classification algorithms in a dataset of matches from Dota 2 (a popular MOBA game). Here's an explanation of the problem: Dota 2 matches are played ...
0
votes
0answers
15 views

Several categories using k-nearest neighbour

Is it possible to have a training data set for the k nearest neighbour with several categories likes 700 and about 10 attributes?
1
vote
2answers
235 views

Doing low-dimensional KNN on a large dataset

I have a simple two-dimensional dataset with columns X1,X2, and [outcome], and I want to try KNN (probably K around 100 or 1000, though ideally CV would be possible). The problem is that my dataset ...
0
votes
1answer
48 views

predict category by using K-NN algorithm having text features

I would like to predict the category of the provided data by using K-NN algorithm. Here is an example of the training data set ...
0
votes
1answer
40 views

Classifying a set of photos to places

I want to cluster photos and map them to places. As input I have Photos with locations (lat, long) Places - some as (imprecise) bounding boxes, some just as points, maybe others as bounding ...