k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

learn more… | top users | synonyms

0
votes
0answers
23 views

Best way to classify a set through a single feature

I need to classify a single dataset through a numeric value. I added below a simple dataset to explain what I need. Restriction: Category has two values: 1 or ...
1
vote
0answers
19 views

What criteria are used to compare feature-based classification techniques?

When comparing feature-based classification techniques, what characteristics about the different processes should be considered? I'm comparing different classification techniques to try to figure ...
0
votes
0answers
24 views

Feature Selection for look alike modeling using k-NN

I have a list of items and various parameters for each items. For each item on my list i need to identify items which are similar to the item from my whole population . I am planning on using K-NN ...
0
votes
1answer
53 views

All nearest neighbors in high dimensional space

Suppose I have a very large binary matrix representing $n$ customers and the $m$ products they bought, with $n$ and $m$ both rather large (in the order of millions). The matrix is also pretty sparse. ...
1
vote
1answer
53 views

How to define the maximum k of the kNN classifier?

I am trying to use kNN classifier to perform some supervised learning. In order to find the best number of 'k' of kNN, I used cross validation. For example, the following codes load some Matlab ...
6
votes
1answer
192 views

Can the curse of dimensionality be solved by changing the metric tensor?

My (not too deep) understanding of the curse of dimensionality that affects a classification algorithm, such as k-Nearest Neighbor, is that at higher dimensions the 'sparsity' of euclidean space kicks ...
1
vote
1answer
54 views

How to interpret the number of k in k-nearest-neighbour classifier?

I have done some classification work using a k-nearest-neighbour classifier (kNN). And the classification performance is evaluated using cross-validation method. Some testing code from Matlab Help are ...
1
vote
0answers
25 views

regression with kNN on dataset with categorical variables

I am trying to train a regression model for dataset with 500k observations and 3 features. The features are categorical and have 50, 50 and 100 levels. Is (generally) kNN appropriate for this kind of ...
1
vote
0answers
29 views

Using KNN (with contributing weights) to calculate ranking

Sorry about the noob question, we tried looking through the answers but couldn't make sense of it. We are very basic with our math/stat knowledge, so please bear with us if this makes less relevant ...
0
votes
0answers
14 views

Implementation of prototype selection in application to nearest neighbor classifier

I am looking for an implementation of prototype selection algorithm and it is highly preferable that implementation would be in Python. I am using sikit-learn, but there is none.
0
votes
0answers
12 views

Increase Recall Rate for SIFT

Is there a better way to increase Recall Rate when using SIFT features? I am thinking a way to replace the NN1/NN2 ratio to account for slightly distorted objects. Moving towards clustering and using ...
1
vote
1answer
46 views

Using KNN for 2 Dimentional data

I am dealing with sentiment results of web articles. Sentiment is represented with two int values, +ve and -ve. For given article "xyz" I am getting different sentiments while testing at different ...
0
votes
0answers
63 views

How does knnimpute of the preprocess function work?

I am new to R and I use a script I do not completely understand. It preprocesses a dataset for data mining. At one point, the data (stored in fil) should be ...
0
votes
2answers
73 views

How to obtain the class conditional probability when using KNN classifier?

Using the KNN classifier, given a test data-point $x$, how do we get the probability of membership of $x$ to each class $y_i$, that is the probabilities $P(y_i | x)$ for $i = 1, 2, .., n$ (where $n$ ...
6
votes
3answers
136 views

Why is KNN not “model-based”?

ESL chapter 2.4 seems to classify linear regression as "model-based", because it assumes $f(x) \approx x\cdot\beta$, whereas no similar approximation is stated for k-nearest neighbors. But aren't both ...
0
votes
0answers
42 views

Using kknn regression function with NAs

I have been lurking here for awhile and now have a question I hope to get answered! I am wondering how to get nearest neighbor algorithms to deal with NAs most effectively. I am dealing with a data ...
1
vote
0answers
22 views

k nearest neighbors with repeated points

An example illustrates my question. Here I use the FNN package in R to find the indices of the 400 nearest points for each ...
1
vote
2answers
83 views

Does K-means incorporate the K-nearest neighbour algorithm?

I was watching this tutorial on K-means clustering and from what I understand K-means is: Randomly generate the centroids for k clusters Create a classification model dividing into k regions (Do we ...
0
votes
0answers
68 views

Standard score (z-transformation) normalization or 0-1 normalization, which one is better for k-NN?

I am not much experienced in data mining, but I know that I should normalize my data before running k-NN classifier on it, to have reasonable results. But I found out, that there are many methods to ...
0
votes
0answers
29 views

Are there some risks connected with using k-NN with k=1 by F=0.77?

I am not very expirienced in the field of machine learning. For now I model some binary classifier using k-NN. I experimented a bit with different values of k. In some cases the best one was with ...
0
votes
0answers
29 views

binarization of variable - experimental threshold choice. Is it good approach?

I have some ratings averages values from 1 to 5(users were rating on 1,2,3,4,5 scale). I would like to split them into two classes: credible, ...
3
votes
1answer
111 views

Metric for nearest neighbor method

Is there a requirement that the measure used in Nearest Neighbor methods be a proper metric distance? What will happen if I use an arbitrary function (e.g., one that does not satisfy the triangle ...
1
vote
0answers
119 views

Using KNN for prediction, how should I normalize my data?

Is it better to constrain the data to a range, say [0,1], or to force a mean of 0 and sd of 1? Why? Does the type of input data matter (I'll be using both continuous and categorical variables)?
4
votes
2answers
344 views

How to approach forecasting time-series data

I'm statistics newbie and any help in picking a good method to analyze the data that I have would be very welcome: We have a customer that has an active Facebook page, that gets posted on regularity. ...
3
votes
1answer
146 views

Using the eigenvalues from PCA in k-nearest-neighbours

I'm quite new to this StackExchange, only been a lurker till now, but my StackOverflow fellows have said you'd be the best people to ask about this. Anyway, enough introduction. I'm using the ...
0
votes
1answer
80 views

How to find whether the k nearest neighbors of an instance belongs to same class or not

I am working on imbalanced datasets which have two classes: majority and minority class. Here I want to find whether the minority class examples are majority class. How can I do that? I have ...
0
votes
2answers
50 views

algorithm for finding similar items

I have a group of items S, say there are 100 items in S, and I know some features of these items, i.e., color, size, ...Now, I have another group of items P, say there are 10000 items in P. What can I ...
3
votes
2answers
128 views

Choice of distance metric when data is combination text/numeric/categorical

I have a large table of attributes of different real-world movie theaters. I have classified them by the "true" physical entity to which they belong, so that there may be multiple records for a given ...
4
votes
0answers
129 views

Estimating model error in $k$-nearest neighbours with strongly spatially autocorrelated training data

In the palaeoclimate world, palaeoecologists have used spatial training sets of say sea-surface temperture (SST) and related this to micro-organisms living at the locations where SST was measured. A ...
6
votes
1answer
143 views

Knapsack problem with uncertain profits

I am trying to find the optimal strategy for a game where the goal is to pick a number of items where the profit is uncertain but the weights are set. Each round of picking is a specific point in ...
0
votes
0answers
56 views

K-Nearest-Neighbors Probability Distributions

I have a question about K-nearest-neighbors and whether it is possible to output more estimates than just the weighted estimate that the R package KKNN package gives you. I am predicting sports ...
1
vote
0answers
44 views

Nearest neighbor methods

Given a training set $\{(x_1, y_1), \dots, (x_n, y_n)\}$, consider the following: $$\hat{Y}(x_i) = \frac{1}{k} \sum_{x_i \in N_{k}(x)} y_i$$ Suppose $k=3$. Is this formula saying that we pick the $3$ ...
0
votes
0answers
58 views

Can ITML overfit?

When training using Information Theoretic Metric Learning (ITML), should I be concerned that the algorithm will overfit the data? If yes, what is the best way to avoid it? ...
4
votes
2answers
2k views

KNN imputation R packages

I am looking for a KNN imputation package. I have been looking at imputation package (http://cran.r-project.org/web/packages/imputation/imputation.pdf) but for some reason the KNN impute function ...
2
votes
1answer
54 views

Categorizing transportation modes with k-nearest neighbor?

I'm working on a classifier to sort out transportation modes based on certain attributes of an activity. I use a nearest neighbor algorithm like this : 2 example sets of training data (let's assume ...
0
votes
0answers
97 views

How do we get log likelihood of KNN?

For getting log-likelihood values, I am using R AIC() method. Although I can get the log-likelihood values of linear regression models, getting the following error for when I applied R AIC() method on ...
0
votes
1answer
1k views

Normalized root mean squared error (NRMSE) vs root mean squared error (RMSE)

The response values in my data set (100 data points) are all positive integers (should not be either negative or zero values). I have developed two statistical models: Linear Regression (LR) and K ...
0
votes
0answers
60 views

Large scale k nearest neighbor search

For example we have n samples with vector length k (n>>k). And we can't load this matrix in RAM at once. Is there any solutions for large scale nearest neighbor search? any libs suitable for this? ...
0
votes
0answers
80 views

Overfitting in K-NN and Decision Trees?

To avoid over fitting for K-NN could you increase the value of K to reduce anomalous results etc. However, if the value of K is very large with respect to a sample, would this also incur in over ...
0
votes
0answers
237 views

Image classifier in python for few samples

I have 150 pictures that represent archeological signs and 5 categories to which they belong. These pictures have features like circularity, roughness and elongation that are expressed as continuous ...
0
votes
0answers
35 views

Solving K-nearest Neighbor Density estimation

I'm trying to do kn-nearest neighbor estimation for 1, 2 and 3 dimensions. I know the formula is $ p_n(x) = \frac{(k_n/n)}{V_n} $. Looking at the problem we are supposed to plot a density estimate ...
3
votes
2answers
85 views

In general how do you set K in K-NN?

As the title suggests, how should you set K in K-Nearest Neighbours? Is it just a case of lower values of K are more susceptible to over-fitting and larger values of K are likely to give a more ...
12
votes
2answers
3k views

What are the main differences between K-means and K-nearest neighbours?

I know that k-means is unsupervised and is used for clustering etc and that k-NN is supervised. But I wanted to know concrete differences between the two?
0
votes
1answer
108 views

How would I use k-nearest neighbours to solve this problem?

I am not too sure how to use K-NN to calculate the error on this data set (as shown below). Any help would be appreciated. Source: http://imgur.com/WqbsYDu
4
votes
2answers
139 views

Nearest Neighbor Algorithm for Circular dimensions

Is there an algorithm for fast nearest neighbor search of circular dimensions? e.g., For a dimension based on "hour of day", a KD-tree would place 00:01 and 23:59 far apart. But the proper distance ...
0
votes
1answer
138 views

How to scale new datas when a training set already exists

Here is what I have : A scaled training set, with labels. Segmented images, from which I extract new vectors to classify. My classifier is a KNN which would have obviously been trained using my ...
2
votes
0answers
159 views

How to use k nearest neighbours for binary classification with unbalanced classes?

I have relatively large (100k items) dataset which I need to split in two groups. So far I've tried knn and the results are not good mainly because I have disproportion in my training data: 90% of ...
1
vote
1answer
49 views

nearest neighbors degrees of freedom

For polynomial fitting with a polynomial of degree $n$, we have $n$ degrees of freedom. Is there a similar concept for $k$ nearest neighbors? Is there any way to compare the degrees in general? I come ...
0
votes
0answers
41 views

How to preform 1NN with single centroid per class in SAS?

I've computer a single centroid per class using PROC fastclus in SAS, ...
3
votes
3answers
658 views

Dealing with ties, weights and voting in kNN

I am programming a kNN algorithm and would like to know the following: Tie-breaks: What happens if there is no clear winner in the majority voting? E.g. all k nearest neighbors are from different ...