k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

learn more… | top users | synonyms

0
votes
0answers
11 views

Binary classification with KNN

I post here because I don't know how to improve the performance of my binary KNN. The problem is that I have 99.8% Specificity and only 82% Sensitivity, but I'd rather have more Sensitivity than ...
0
votes
0answers
26 views

KNN - does it use centroids?

I'm really confused now having read so many articles on KNN, I can't help but think I'm missing the obvious. Let's say I have persons P1 and P2, P3 are represented with attributes of height, weight ...
0
votes
3answers
44 views

Is kNN best for classification?

I wanted to know if kNN might produce the best result for classification? Since, it is not model based, it does not loose any detail and compares every training sample to give the prediction. Hence ...
0
votes
0answers
15 views

Why SVM with RBF is similar to KNN with prototypes search?

I explored similar questions and everything I can see is that both kNN and RBF are non-parametric methods to estimate the density of probability of your data. However, I am not sure if this has ...
0
votes
0answers
15 views

How to use Similarity Measure in K-nearest neighbor Classification?

I have a similarity measure just like cosine. How can i use that similarity measure in traditional k-nn classification? Please provide some literature review (research papers) details which i should ...
3
votes
1answer
56 views

Excluding the scatter points from a feature

I have a set of data points that are supposed to sit on a locus and follow a pattern, but there are some scatter points from the main locus that cause uncertainty in my final analysis. I would like to ...
1
vote
1answer
31 views

Classification tips for a begginer

I'm doing a graduation work that involves applying Classification algorithms in a dataset of matches from Dota 2 (a popular MOBA game). Here's an explanation of the problem: Dota 2 matches are played ...
0
votes
0answers
12 views

Several categories using k-nearest neighbour

Is it possible to have a training data set for the k nearest neighbour with several categories likes 700 and about 10 attributes?
1
vote
2answers
103 views

Doing low-dimensional KNN on a large dataset

I have a simple two-dimensional dataset with columns X1,X2, and [outcome], and I want to try KNN (probably K around 100 or 1000, though ideally CV would be possible). The problem is that my dataset ...
0
votes
1answer
30 views

predict category by using K-NN algorithm having text features

I would like to predict the category of the provided data by using K-NN algorithm. Here is an example of the training data set ...
0
votes
1answer
33 views

Classifying a set of photos to places

I want to cluster photos and map them to places. As input I have Photos with locations (lat, long) Places - some as (imprecise) bounding boxes, some just as points, maybe others as bounding ...
0
votes
0answers
25 views

Method to select meaningful features for nearest neighbor classification

i try to perform some k nearest neighbor classification in R. That for i want to select the most meaningful features to deal with the curse of dimensionality. I have already decided to use Mahalanobis ...
2
votes
3answers
207 views

Does k-NN with k=1 always implies overfitting?

I found somewhere such statement, but on the other hand in some sources I found, that it is ok. What about risk of overfitting while using 1-NN in binary classification problem where explanatory ...
0
votes
1answer
49 views

advantage of euclidean distance for classification

Has euclidean distance any advantage in compare to another distance based methods like Manhatan distance or Maximum difference metric?
5
votes
2answers
157 views

Is KNN a discriminative learning algorithm?

It seems that KNN is a discriminative learning algorithm but I can't seem to find any online sources confirming this. Is KNN a discriminative learning algorithm?
1
vote
2answers
153 views

How to deal with unbalanced data

I'm doing data analysis with a dataset of 11795 data points (with 88 features). 85% (9973 points) of these data points correspond to data points belonging to class 1, 5% (589 points) belong to class 2 ...
0
votes
0answers
20 views

sample size to estimate the best k in kNN

I would like to use a kNN classifier. My data set is quite small and include 2 classes having about 200 samples each one. I need to estimate k by using a cross-validation approach. How many samples ...
8
votes
2answers
577 views

Why would anyone use KNN for regression?

From what I understand, we can only build a regression function that lies within the interval of the training data. For example (only one of the panels is necessary): How would I predict into the ...
2
votes
1answer
157 views

Pros of Jeffries Matusita distance

According to some paper I am reading, Jeffries and Matusita distance is commonly used. But I couldn't find much information on it except for the formula below ...
0
votes
2answers
61 views

Regression: what to do with categorical predictors?

I've got a few categorical predictors (like gender,...) and now I want to build regression models. So I've made the categorical predictors numeric by for example: "female" --> 1 and "male" --> 0. But ...
0
votes
1answer
51 views

Is this a correct interpretation of k nearest neighbours?

Given this dataset : name1,name2,distance a,b,1 a,c,5 b,c,8 If k=1 is the following correct : ...
2
votes
0answers
113 views

kozachenko-leonenko entropy estimation

I'm trying to implement the entropy estimation based on the closest neighbour from Kozachenko and Leonenko but I'm facing a problem I can't solve. The idea is to work in a new set ...
0
votes
2answers
54 views

Is this a valid use case for Euclidean distance?

I have a set of points which is a count of links that users have clicked on : ...
0
votes
1answer
30 views

Looking for measurement for nonlinear model

I use KNN regression to train out a model. The model estimates running time of a program based on different inputs, and the output is a single variable, which is time (double type). I want to ...
0
votes
0answers
34 views

Best way to classify a set through a single feature

I need to classify a single dataset through a numeric value. I added below a simple dataset to explain what I need. Restriction: Category has two values: 1 or ...
1
vote
0answers
47 views

What criteria are used to compare feature-based classification techniques?

When comparing feature-based classification techniques, what characteristics about the different processes should be considered? I'm comparing different classification techniques to try to figure ...
0
votes
0answers
85 views

Feature Selection for look alike modeling using k-NN

I have a list of items and various parameters for each items. For each item on my list i need to identify items which are similar to the item from my whole population . I am planning on using K-NN ...
0
votes
1answer
65 views

All nearest neighbors in high dimensional space

Suppose I have a very large binary matrix representing $n$ customers and the $m$ products they bought, with $n$ and $m$ both rather large (in the order of millions). The matrix is also pretty sparse. ...
1
vote
1answer
286 views

How to define the maximum k of the kNN classifier?

I am trying to use kNN classifier to perform some supervised learning. In order to find the best number of 'k' of kNN, I used cross validation. For example, the following codes load some Matlab ...
7
votes
1answer
209 views

Can the curse of dimensionality be solved by changing the metric tensor?

My (not too deep) understanding of the curse of dimensionality that affects a classification algorithm, such as k-Nearest Neighbor, is that at higher dimensions the 'sparsity' of euclidean space kicks ...
1
vote
1answer
116 views

How to interpret the number of k in k-nearest-neighbour classifier?

I have done some classification work using a k-nearest-neighbour classifier (kNN). And the classification performance is evaluated using cross-validation method. Some testing code from Matlab Help are ...
1
vote
0answers
133 views

regression with kNN on dataset with categorical variables

I am trying to train a regression model for dataset with 500k observations and 3 features. The features are categorical and have 50, 50 and 100 levels. Is (generally) kNN appropriate for this kind of ...
1
vote
0answers
57 views

Using KNN (with contributing weights) to calculate ranking

Sorry about the noob question, we tried looking through the answers but couldn't make sense of it. We are very basic with our math/stat knowledge, so please bear with us if this makes less relevant ...
0
votes
0answers
32 views

Implementation of prototype selection in application to nearest neighbor classifier

I am looking for an implementation of prototype selection algorithm and it is highly preferable that implementation would be in Python. I am using sikit-learn, but there is none.
0
votes
0answers
13 views

Increase Recall Rate for SIFT

Is there a better way to increase Recall Rate when using SIFT features? I am thinking a way to replace the NN1/NN2 ratio to account for slightly distorted objects. Moving towards clustering and using ...
1
vote
1answer
59 views

Using KNN for 2 Dimentional data

I am dealing with sentiment results of web articles. Sentiment is represented with two int values, +ve and -ve. For given article "xyz" I am getting different sentiments while testing at different ...
0
votes
0answers
284 views

How does knnimpute of the preprocess function work?

I am new to R and I use a script I do not completely understand. It preprocesses a dataset for data mining. At one point, the data (stored in fil) should be ...
0
votes
2answers
189 views

How to obtain the class conditional probability when using KNN classifier?

Using the KNN classifier, given a test data-point $x$, how do we get the probability of membership of $x$ to each class $y_i$, that is the probabilities $P(y_i | x)$ for $i = 1, 2, .., n$ (where $n$ ...
6
votes
3answers
188 views

Why is KNN not “model-based”?

ESL chapter 2.4 seems to classify linear regression as "model-based", because it assumes $f(x) \approx x\cdot\beta$, whereas no similar approximation is stated for k-nearest neighbors. But aren't both ...
0
votes
0answers
68 views

Using kknn regression function with NAs

I have been lurking here for awhile and now have a question I hope to get answered! I am wondering how to get nearest neighbor algorithms to deal with NAs most effectively. I am dealing with a data ...
1
vote
0answers
28 views

k nearest neighbors with repeated points

An example illustrates my question. Here I use the FNN package in R to find the indices of the 400 nearest points for each ...
1
vote
2answers
154 views

Does K-means incorporate the K-nearest neighbour algorithm?

I was watching this tutorial on K-means clustering and from what I understand K-means is: Randomly generate the centroids for k clusters Create a classification model dividing into k regions (Do we ...
0
votes
0answers
135 views

Standard score (z-transformation) normalization or 0-1 normalization, which one is better for k-NN?

I am not much experienced in data mining, but I know that I should normalize my data before running k-NN classifier on it, to have reasonable results. But I found out, that there are many methods to ...
0
votes
0answers
29 views

Are there some risks connected with using k-NN with k=1 by F=0.77?

I am not very expirienced in the field of machine learning. For now I model some binary classifier using k-NN. I experimented a bit with different values of k. In some cases the best one was with ...
0
votes
0answers
41 views

binarization of variable - experimental threshold choice. Is it good approach?

I have some ratings averages values from 1 to 5(users were rating on 1,2,3,4,5 scale). I would like to split them into two classes: credible, ...
3
votes
1answer
127 views

Metric for nearest neighbor method

Is there a requirement that the measure used in Nearest Neighbor methods be a proper metric distance? What will happen if I use an arbitrary function (e.g., one that does not satisfy the triangle ...
1
vote
0answers
305 views

Using KNN for prediction, how should I normalize my data?

Is it better to constrain the data to a range, say [0,1], or to force a mean of 0 and sd of 1? Why? Does the type of input data matter (I'll be using both continuous and categorical variables)?
4
votes
2answers
770 views

How to approach forecasting time-series data

I'm statistics newbie and any help in picking a good method to analyze the data that I have would be very welcome: We have a customer that has an active Facebook page, that gets posted on regularity. ...
4
votes
1answer
224 views

Using the eigenvalues from PCA in k-nearest-neighbours

I'm quite new to this StackExchange, only been a lurker till now, but my StackOverflow fellows have said you'd be the best people to ask about this. Anyway, enough introduction. I'm using the ...
0
votes
1answer
95 views

How to find whether the k nearest neighbors of an instance belongs to same class or not

I am working on imbalanced datasets which have two classes: majority and minority class. Here I want to find whether the minority class examples are majority class. How can I do that? I have ...