k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

learn more… | top users | synonyms

0
votes
0answers
5 views

Statistical Test to Use with Nearest Neighbor Matching with Replacement?

I'm planning on doing a project to measure the effect of a treatment, using nearest neighbor matching with replacement of control units (i.e. more than one test unit can be matched to the same control ...
0
votes
0answers
20 views

Volume for Parzen window and $k_n$ Nearest Neighbors density estimation

In Parzen window estimate, suppose we want to estimate the density at $x_o$, what we do is raise, say a uniform kernel at $x_o$, find number of points falling in the box, which is $k_n = ...
2
votes
2answers
51 views

How to predict property value using lat/lon?

I have lat/lon and property values for households in a particular region. Format: Lat Lon value 32.2 -98.22 120000 .... Now I have new data of the ...
0
votes
1answer
32 views

Finding N-Most Similar Organizations to a Given Organization

I am looking to provide a model with one data point and have it return the observations most similar to that given data point. I am in the process of developing peer groups from a dataset of 240 ...
0
votes
0answers
7 views

Dealing with high dimensionality in bag of words (BOW) model

I have a large amount of documents of equal size. For each of those documents I'm building a bag of words model (BOW). Number of possible words in all documents is limited and large (2^16 for ...
1
vote
0answers
17 views

How can weighted K-NN decrease accuracy?

I have heard that weighted K-NN improves the accuracy.But in my case, the accuracy decreases.Can anyone suggest why there is a decrease in accuracy?
0
votes
0answers
37 views

Kernel K nearest neighbours with sparse data

I have a big sparse matrix (around 5 million of lines, 20 000 predictors), and I would like to run a kernelized k-NN on it. However, I don't know how to scale the data properly. So far, I have scaled ...
0
votes
2answers
57 views

R - Multivariate K-nearest neighbor outlier detection

I'm trying to implement the algorithm K-nearest neighbor to detect outlier from a multivariate dataset. I don't know how to do it. Could you provide me some example?
5
votes
2answers
168 views

Curse of dimensionality: kNN classifier

I am reading Kevin Murphy's book: Machine Learning-A probabilistic Perspective. In the first chapter the author is explaining the curse of dimensionality and there is a part which i do not understand. ...
1
vote
0answers
16 views

how to understand this neighborhood components analysis model?

I am reading an article with title "neighborhood components analysis" lately. http://papers.nips.cc/paper/2566-neighbourhood-components-analysis.pdf. This article is trying to introduce a linear ...
1
vote
0answers
25 views

Pass custom weight function to 'kknn' model in Caret package

I am working on school project, where I'm trying to implement improvement for weighted kNN in CARET package. I basically need to replace standard 'weight' function used in KKNN model to something more ...
0
votes
0answers
12 views

Quasi-experiment with naturally assembled collectives (classrooms): Propensity score matching or not?

I am planning on running a quasi-experiment (pre-test post-test non-equivalent groups design) that involves two treatment groups that are subjected to different treatments (X+ & X-) and one ...
0
votes
1answer
35 views

What is the final equation used to produce new prediction using kknn on R

I have trained my data using kknn on R and was able to predict on a new data set. However, I'd like to know what the actual final equation is so I can reproduce the prediction manually. My training ...
1
vote
0answers
25 views

Suspicious Amount of Zeros in Confusion Matrix

I have a data set with about 45000 observations and three features. When I apply machine learning classification algorithms like naive Bayes, kNN and SVM I receive a lot of zeros in the resulting ...
3
votes
3answers
412 views

KNN: 1-nearest neighbor

My question is about the 1-nearest neighbor classifier and is about a statement made in the excellent book The Elements of Statistical Learning, by Hastie, Tibshirani and Friedman. The statement is ...
0
votes
0answers
48 views

Using variance in place of standard deviation for z-normalization

I'm implementing a 1-nearest neighbor (with dynamic time warping as the distance measure) classification algorithm on a severely constrained embedded platform with no FPU, so we're doing fixed point ...
4
votes
3answers
141 views

Is it feasible to use k-Nearest Neighbours to identify text language?

I have seen various language identification libraries that claim to use naive Bayes classifier for text language identification, like CLD2 and language detector, but not any library that uses other ...
1
vote
0answers
14 views

Finding multiple records of the same person when the name cannot be trusted

I have a list of 100 of events, and each event has a list of thousands of persons. Some persons went to multiple events, and I would like to select all the events a person went to. I could just look ...
1
vote
0answers
128 views

Support Vector Machines vs KNN

It was my understanding that in a separable case, SVMs produce the best separation possible and therefore will always produce the same or a better classification rate compared with say, 1NN, ...
2
votes
1answer
94 views

How does Scikit Learn resolve ties in the KNN classification?

I have a multi-class classification problem, in which I'm using Scikit Learn's k nearest neighbour classifier, (5 classes), which means that an odd number for k won't prevent classification ties. So ...
5
votes
2answers
78 views

How can increasing the dimension increase the variance without increasing the bias in kNN?

My question is about understanding Figure 2.8 in The Elements of Statistical Learning (2nd edition). The topic of the section is how increasing dimension influence the bias/variance. I can roughly ...
3
votes
1answer
88 views

some inference about k-NN algorithms for better understanding? [closed]

I ran into some facts make me confusing. for k-NN classifier: I) why classification accuracy is not better with large values of k. II) the decision boundary is not smoother with smaller ...
0
votes
1answer
177 views

Locally weighted regression VS kernel linear regression?

I am trying to make it clear the relationship of the listed three methods. According to my understanding kernel regression means : the weight vector W lies in the space spanned by training data. $$ ...
1
vote
1answer
307 views

Buiding Ensemble model

I'm new to ensemble model. Suppose I've KNN models like this - (in R) library(class) ...
3
votes
1answer
116 views

Two broad categories of dimensionality reduction

As a starter in dimensionality reduction (DR), I recently acquired the following understanding. There are two very broad categories of DR techniques: We can compute an analytic form of mapping from ...
0
votes
1answer
25 views

Asymmetric distance measure in k-NN classifier?

What is the problem with an asymmetric distance measure in k-NN classifier? I think it will not cause problem, so long as I compute the distance consistently, say always from ...
3
votes
1answer
88 views

Cumulative match score

I have seen loads of graphs in papers of cumulative match scoring, but I can't find any information about what it means, or how it is created. A context that would be useful to see the explanation ...
0
votes
1answer
53 views

time series based classification

I want to classify some data. Basically the data is time series in nature. The target variable is categorical. I know there are so many algorithms for predicting the time series model. However, I have ...
0
votes
0answers
40 views

For time series classification, how can k nearest neighbors outperform other models?

Suppose you have a collections of time series data $Y^1,\ldots,Y^N$, $Y^i = Y^i_1,\ldots,Y^i_T$. Your training data consists of labels for some of these $Y^i$, and you wish to infer labels for the ...
0
votes
0answers
15 views

What does choice of metric say about the data?

I am using KNN method. I have three choices of metric (Euclidean, Manhattan and Chebyshev). The Error rate was minimum when I used Chebyshev distance. What does it say about the distribution data?
1
vote
0answers
64 views

Bias and variance of a naive bayes classifier and KNN classifier

After reading the paper by J. Friedman, ”On bias, variance, 0/1-loss, and the curse-of-dimensionality,” Data Mining and Knowl- edge Discovery, 1997. I would like to estimate both bias and variance ...
1
vote
0answers
85 views

Python kNN vs. radius nearest neighbor regression

Python offers two nearest neighbor regressions: radius nearest neighbor and k-nearest neighbor. I'm trying to figure out a few things: 1. Under which circumstances would each be preferable? 2. How do ...
0
votes
0answers
43 views

How can I use the output of KODAMA to predict unknown data points?

I can use KODAMA to create a model that classifies input data into two groups by setting the W vector to indicate the group and fix to a vector of all ...
1
vote
0answers
33 views

Does centering or mean normalizaiton alone every help in feature scaling?

In feature scaling, one way is to subtract the mean (centering) and then divide by the standard deviation for all data points. Suppose we just centered the data and didn't divide by the standard ...
0
votes
0answers
11 views

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor?

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor when no test set is available, --when the input vectors are unknowns? Here's my problem, I have a massive ...
2
votes
1answer
192 views

How to handle data normalization in kNN when new test data is received

I had a discussion with my colleagues about the following problem: Lets say we have 100 points of labeled data and we are using $k$-nearest neighbor method for prediction. So our data looks like ...
6
votes
1answer
196 views

VC-Dimension of k-nearest neighbor

What is the VC-Dimension of the k-nearest neighbor algorithm if k is equal to the number of training points used? Context: This question was asked in a course I take and the answer given there was ...
0
votes
0answers
32 views

data treatment before or after train/test sets split?

I have a variable in my data with NA values and I want to apply knn input. Should I do it before or after split the data in train and test set? If I do it after, each set will only use the values in ...
3
votes
1answer
127 views

What does the k-value stand for in a KNN model?

What is the k-value in a KNN classification model? Is K the number of Clusters?
1
vote
0answers
29 views

KNN converging to regression function proof (ESL page 19)

In Elements of Statistical Learning, page 19, it says I am confused what it means by "under mild regularity condition" and how one can prove the statement $\hat{f}(x) \rightarrow E(Y|X = x)$ under ...
0
votes
0answers
18 views

Smoothing of a 2D Empirical Distribution

I have a number of data points $\theta \in \mathbb{R}^2$ with corresponding values $x \in \mathbb{N}$. I am assuming the $x$ are realisations from a distribution $f(X | \theta)$. Given I have a lot ...
1
vote
1answer
187 views

If Manhattan distance always performs better on a dataset…what does it mean?

I'm analyzing my dataset using kNN. I experimented with various distance functions but Manhattan seems to perform better in terms of lowest RMSE over various values of k. I've read a bit about ...
5
votes
1answer
165 views

Explanation of formula for median closest point to origin of N samples from unit ball

In Elements of Statistical Learning, a problem is introduced to highlight issues with k-nn in high dimensional spaces. There are $N$ data points that are uniformly distributed in a $p$-dimensional ...
1
vote
0answers
19 views

Can KNN theoretically be less accurate than pattern averaging?

My task is pattern recognition. I need to classify 2D matrices into an arbitrary number of classes. The question is: For pattern classification, could k nearest neighbours algorithm ever be less ...
0
votes
0answers
16 views

What is the cost of finding neighbors in eps radius?

Finding neighbors in eps radius a sample point is called region query. There are data structures reduce cost of such queries. These structures are also used to give k-nearest neighbors of given sample ...
0
votes
0answers
170 views

Naive Bayes Nearest Neighbor (NBNN) implementation problems in MATLAB

I'm currently trying to classify the CIFAR-10 image dataset. I cam across a number of papers praising the the results from a non-parametric approach called Naive Bayes Nearest Neighbors. It uses SIFT ...
3
votes
3answers
240 views

Choosing optimal K for KNN

I performed a 5-fold CV to select the optimal K for KNN. And it seems like the bigger K gets, the smaller the error... Sorry I didn't have a legend, but the different colors represent different ...
1
vote
0answers
147 views

Distance measure for categorical attributes for k-Nearest Neighbor

For my class project, I am working on the Kaggle competition - Don't get kicked The project is to classify test data as good/bad buy for cars. There are 34 features and the data is highly skewed. I ...
0
votes
0answers
17 views

Find input image (ID,passport) in imagesDB based on similarity

I would like to decide if image is exists on DB images (pictures of IDs,passport,Stu. card,etc) I thought of KNN alghorithem that will plot the K closest images. Options for distance metric: 1) sum ...
0
votes
1answer
61 views

Are there optimization methods for k-NN parameter $k$?

Currently I am just go through from the min to the max, and determine $k$ by the performance. I am wondering if there's optimization approaches for selecting $k$? I am aware of there's question in ...