k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

learn more… | top users | synonyms

0
votes
0answers
17 views

Evaluating performance Neural Network embeddings in kNN classifier

I am solving a classification problem. I train my unsupervised neural network for a set of entities (using skip-gram architecture). The way I evaluate is to search k nearest neighbours for each point ...
0
votes
0answers
10 views

Difference between Nearest Neighbour and Nearest Centroid

I'm trying to understand the difference between Nearest neighbour classifiers and Nearest centroid classifier. Using the nearest neighbour, one selects a data point ...
0
votes
1answer
47 views

Time series analysis for predicting a binary outcome

I'm fairly new to time series analysis. I want to analyze two series of variables in a span of time to predict a binary outcome. For example i collect data over time at my home of two variables: ...
0
votes
0answers
32 views

How to draw a 3 nearest neighbour decision boundary

I have an exam tomorrow, and I can't seem to get my head around how to do this, nor can I find any information online for this particular case of a 3NN decision boundary. We are presented with sets of ...
1
vote
1answer
25 views

Cross Validation and Nearest Neighbors

What is the best way to approach multiple-fold cross validation for a 1 nearest-neighbor model used for prediction? A common approach to cross validation is to, for example, split the dataset into ...
1
vote
0answers
21 views

application of 1-nearest-neighbor

In the book The elements of statistical learning by Trevor Hastie, there is a sentence (on page 17) saying : In fact 1-nearest-neighbor, the simplest of all, captures a large percentage of the ...
1
vote
1answer
121 views

minimizing total travel time [closed]

Green circles are salesman and A-B-C are destination points. I need to move green circles so that total travelled time should be minimum and each salesman should go to a different point. So ideal ...
0
votes
1answer
23 views

kNN speed up using triangle inequality

can someone explain how the triangle inequality helps speed up kNN? I understand the general principle of the triangle inequality, however I don't see how a lower bound on $d(x_1, x_2)$ would help ...
0
votes
0answers
11 views

Dynamic Time Warping finds erroneous similarity between time series

I am implementing 1NN using DTW as the distance measurement. It finds erroneous similarity between two time series when they are actually not similar. To give an example, let's suppose a & b are ...
0
votes
1answer
20 views

how to find the nearest neighbor of a sparse vector

I have about 500 vectors,each vector is a 1500-dimension vector, and almost every vector is very sparse-- I mean only about 30-70 dimension of the vector is not 0。 Now, the problom is that here is a ...
1
vote
2answers
50 views

Using ML approaches to build a recommender engine for sales team

I work at a startup as a developer, but I wanted to help out our sales team with running some ML algorithms on the data. A bit of context: Most of our revenue comes from ad purchases, so in a ...
0
votes
0answers
49 views

Is this wikipedia article about KNN contradicting itself regarding “non-parametric”?

I understood that KNN (K-Nearest-Neighbors) was non-parametric, after reading the beginning of the wikipedia article here: ...
0
votes
1answer
32 views

Is it necessary to transform data before k nearest neighbors?

I am trying to follow a method of filling in null values proposed by AirBnB. They have you transform the data before computing distances for KNN. Why is it necessary to transform data? Is its ...
1
vote
1answer
71 views

Find K-nearest neighbour with custom distance metric

I am working on finding similar items. Each item has a representation as a vector of features. Instead of using one kind of distance metric for each feature like "ëuclidean" distance. I want a mixture ...
0
votes
1answer
17 views

Applying SMOTE and PCA to high dimensional data giving low accuracy

I have a high dimensional datasets of around 2300+ columns. The dataset consist of two class labels of which one is extremely biased and occurs less than 10%. I looked at the various algorithms and ...
3
votes
1answer
78 views

Does Dimensionality curse effect some models more than others?

The places I have been reading about dimensionality curse explain it in conjunction to kNN primarily, and linear models in general. I regularly see top rankers in Kaggle using thousands of features on ...
0
votes
0answers
7 views

What does $R^2$ mean in the context of K-Nearest Neighbors algorithm?

Googling the meaning of $R^2$ gives the following: "R-squared is a statistical measure of how close the data are to the fitted regression line. It is also known as the coefficient of determination, ...
1
vote
0answers
9 views

Can I prove lack of separability for my problem for which I could not find a good classifier?

I'm not a statistician "by birth" so I would need some validation from actual experts on my ideas for the following problem. My question could sound either murky or obvious, but here it goes. I have ...
0
votes
0answers
14 views

Distances to nearest neighbours of each classes

In the forum post summing up the winning solution for the Kaggle Otto product classification competition, they mention creating a feature which is the "Distances to nearest neighbours of each ...
0
votes
1answer
27 views

error rates of knn minimal for k=1

I am trying to find the best parameter $k$ for a nearest neighbour classifier using cross validation for some datasets. After computing and plotting the error rates, I noticed some strange behaviour ...
0
votes
0answers
10 views

How to find similar documents with mixed features

I am working on finding a similar documents problem. I have 3 sets of features ...
0
votes
0answers
11 views

Any algorithm for knn average?

I have a set of 10 features in a dataset corresponding to member purchase behaviour. I want to use knn to determine nearest neighbours for each member from test dataset on the train dataset based on ...
0
votes
1answer
23 views

k-nn classifier, suspect results, maybe underfitting?

I am using the k-nn classifier in Weka to experiment on the "audiology" dataset. I am using 10-fold validation, and recording average error rates for 1 <= k <= 200. I am trying three different ...
0
votes
1answer
63 views

Influence of Normalization and PCA on Distance Metrics

After standardizing my dataset, I perform a principal component analysis. Then I do a nearest neighbour search. I observed then after performing the PCA, even though I kept (for testing purposes) all ...
1
vote
0answers
76 views

Detection of outliers in 3 dimensions

I have a data set that has x,y,z variables. z is sampled data based on location (x,y). How can I detect potential outliers in z-value, thus corresponding to either, a particular (x,y) location, or a ...
0
votes
0answers
24 views

R: Is it possible to retrieve a KNN model's predicted Y at a given point?

I've been asked to compare the predicted value of a GLM fit at a specific point to that of a KNN fit at the same point. However I can't find any code or functions to do this with KNN models. Is it ...
1
vote
2answers
88 views

Permutation of input features of SVM, simple logistic and KNN classifiers??

I have come across a journal article, with an impact factor higher than 2.8, in which a very strange training procedure was performed. Since I consider myself a beginner, and since the article is ...
1
vote
1answer
168 views

Why KNN is a non linear classifier ?

How do we decide if a classifier is linear or non linear ? What property/characteristic makes a classifier linear or non linear ? Eg: Why SVM is a linear classifier ? Why Logistic Regression is ...
0
votes
0answers
27 views

Hierarchical K-Means structure and nearest neighbor search

I have just recently begun looking into data structures and their use. The two that stood out the most were k-d tree and Hierarchical K-Means. K-d tree was quite straight forward with search as it's ...
1
vote
1answer
60 views

What's the best way to find KNN by hand?

Lets say I'm given the following and need to find 'use' KNN to predict the class label of record 15 and know beforehand that k is set to 3. What are the proper steps, regardless of table, or label or ...
0
votes
1answer
33 views

Exact Procedure for KNN classification

I want to know the exact procedure involved in KNN classification. I understand the bigger picture but I miss the details to implement. I have 3 pieces of data: Train, Validate and Test. 1) Suppose ...
5
votes
1answer
51 views

What is the purpose of row normalization

I understand the reasoning behind column normalization, as it causes features to be weighted equally, even if they are not measured on the same scale - however, often in the nearest neighbour ...
1
vote
0answers
15 views

Reference on analysis of $k$-nearest neighbor density estimation

I am looking for pointers to the analyses of the $k$-nearest neighbor density estimator. In particular, for a fixed $k$, I would like to find the derivation of the mean and the variance of the KNN ...
0
votes
1answer
19 views

How to compute the unconditioned density in $1NN$ classier?

Suppose I have $50$ training points $x_1$, $x_2,\ldots,x_{50}$ and they are distributed via bimodal Gaussian on real line. Now, given a new point, for $1NN$, I am trying to find a interval around $x$ ...
0
votes
0answers
19 views

Help on understanding the asymptotic error for 1-NN classifier

Consider a $1-NN$ classifier. Assume the labels can only be $A_1$ and $A_2$. Let $N$ be number of training points. For a test point $x$, denote by $n_x$ the nearest neighbor or $x$. Then we have the ...
2
votes
1answer
118 views

How to plot visualization for multi-label k-Nearest Neighbor?

I am studying multi-label learning methods, where for a given observation, you can assign more than one (a set of) target labels. One example is multi-label k-Nearest Neighbor. I am seeking a way to ...
2
votes
1answer
71 views

KNN with bagging in R

How to implement bagging with KNN using R in order to reduce the variability? This is the R code that I use for KNN ...
0
votes
1answer
33 views

Learning Curves: Why do people use the holdout set method for Decision Trees and K-NN, but RMSE for Neural Networks?

Based on this question and this link, it seems like people generally use RMSE to understand error and generate learning curves when analyzing a neural network model. But it seems like people ...
0
votes
0answers
95 views

Difference b/w KNN and Decision Tree

What are the differences between KNN classifier and Decision tree classifier? How do one choose between them for solving a classification problem?
2
votes
1answer
105 views

Can Bootstrap aggregation be used with knn classifier?

I am trying to reach a stable model for my classification problem and I am using KNN for classification. In order to improve the accuracy I want to use bootstrap aggregation. In MATLAB I found a ...
2
votes
1answer
51 views

Do you merge your validation and training sets? [duplicate]

I'm currently implementing a K-Nearest Neighbours model and I'm at the stage of splitting up the datasets for cross-validation. I understand the need for the Training, Validation and Test sets, ...
0
votes
0answers
34 views

Using KNN or Clustering techniques to increase data sample size

had a question regarding using KNN or clustering techniques to 'pad' smaller data sets with similar data points. Say, I have some data, a modified and simplified snippet of which looks like this: ...
0
votes
1answer
122 views

Cluster Analysis on GPS data - Assigning GPS coordinates to core groups

I'm trying to figure out a way to assign GPS coordinates to core GPS values. For example, I've got tons of store locations (with long & lat coordinates) and I'd like to group them to one of x ...
2
votes
1answer
52 views

Does a radial basis function network work in high dimensions?

It seems that a single-layer radial basis function network with normalized weights is the same thing as kernel smoothing (see e.g. Haykin Neural Networks: a Comprehensive Foundation, Section 5.12). ...
0
votes
0answers
15 views

Statistical Test to Use with Nearest Neighbor Matching with Replacement?

I'm planning on doing a project to measure the effect of a treatment, using nearest neighbor matching with replacement of control units (i.e. more than one test unit can be matched to the same control ...
1
vote
0answers
64 views

too many ties in knn? how to solve this problem

I use the knn model to train my data and then eliminate accuracy via cross-validation, but when I use the following code, I get the error: ...
0
votes
0answers
38 views

Prediction intervals for kNN regression

I would like compute prediction intervals for predictions made by kNN regression. I can't find any explicit reference to confirm, so my question is - is this approach to computing prediction intervals ...
0
votes
0answers
60 views

Volume for Parzen window and $k_n$ Nearest Neighbors density estimation

In Parzen window estimate, suppose we want to estimate the density at $x_o$, what we do is raise, say a uniform kernel at $x_o$, find number of points falling in the box, which is $k_n = ...
2
votes
2answers
104 views

How to predict property value using lat/lon?

I have lat/lon and property values for households in a particular region. Format: Lat Lon value 32.2 -98.22 120000 .... Now I have new data of the ...
0
votes
1answer
58 views

Finding N-Most Similar Organizations to a Given Organization

I am looking to provide a model with one data point and have it return the observations most similar to that given data point. I am in the process of developing peer groups from a dataset of 240 ...