Tagged Questions

The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
20 views

Large scale k nearest neighbor search

For example we have n samples with vector length k (n>>k). And we can't load this matrix in RAM at once. Is there any solutions for large scale nearest neighbor search? any libs suitable for this? ...
0
votes
0answers
37 views

Overfitting in K-NN and Decision Trees?

To avoid over fitting for K-NN could you increase the value of K to reduce anomalous results etc. However, if the value of K is very large with respect to a sample, would this also incur in over ...
0
votes
0answers
40 views

Image classifier in python for few samples

I have 150 pictures that represent archeological signs and 5 categories to which they belong. These pictures have features like circularity, roughness and elongation that are expressed as continuous ...
0
votes
0answers
16 views

Solving K-nearest Neighbor Density estimation

I'm trying to do kn-nearest neighbor estimation for 1, 2 and 3 dimensions. I know the formula is $ p_n(x) = \frac{(k_n/n)}{V_n} $. Looking at the problem we are supposed to plot a density estimate ...
3
votes
2answers
59 views

In general how do you set K in K-NN?

As the title suggests, how should you set K in K-Nearest Neighbours? Is it just a case of lower values of K are more susceptible to over-fitting and larger values of K are likely to give a more ...
1
vote
1answer
83 views

What are the main differences between K-means and K-nearest neighbours?

I know that k-means is unsupervised and is used for clustering etc and that k-NN is supervised. But I wanted to know concrete differences between the two?
0
votes
1answer
66 views

How would I use k-nearest neighbours to solve this problem?

I am not too sure how to use K-NN to calculate the error on this data set (as shown below). Any help would be appreciated. Source: http://imgur.com/WqbsYDu
0
votes
0answers
86 views

Clustering with “MCLUST” in R

I have worked with R for just a few weeks. I have to use "mclust" for clustering my data (small dataset) and specify the (right) number of clusters and plot it. This is what I've done sofar (see ...
3
votes
2answers
70 views

Nearest Neighbor Algorithm for Circular dimensions

Is there an algorithm for fast nearest neighbor search of circular dimensions? e.g., For a dimension based on "hour of day", a KD-tree would place 00:01 and 23:59 far apart. But the proper distance ...
0
votes
1answer
101 views

How to scale new datas when a training set already exists

Here is what I have : A scaled training set, with labels. Segmented images, from which I extract new vectors to classify. My classifier is a KNN which would have obviously been trained using my ...
0
votes
0answers
40 views

Reformulate binary classification: Relative penalty of false positives versus false negatives

I have a training data for a set of insurance claims that I used to train multiple models within R (1. Binomial Logistic regression 2. Naive Bayes 3. k nearest neighbor algorithm) The binomial ...
2
votes
0answers
76 views

How to use k nearest neighbours for binary classification with unbalanced classes?

I have relatively large (100k items) dataset which I need to split in two groups. So far I've tried knn and the results are not good mainly because I have disproportion in my training data: 90% of ...
1
vote
0answers
26 views

nearest neighbors degrees of freedom

For polynomial fitting with a polynomial of degree $n$, we have $n$ degrees of freedom. Is there a similar concept for $k$ nearest neighbors? Is there any way to compare the degrees in general? I come ...
0
votes
0answers
30 views

How to preform 1NN with single centroid per class in SAS?

I've computer a single centroid per class using PROC fastclus in SAS, ...
3
votes
3answers
238 views

Dealing with ties, weights and voting in kNN

I am programming a kNN algorithm and would like to know the following: Tie-breaks: What happens if there is no clear winner in the majority voting? E.g. all k nearest neighbors are from different ...
1
vote
0answers
21 views

Relative to a Population: Statistically Valid Method for Categorizing Something as “Other”?

I am working similarity searching of an HTTP Request relative to the last N number of days of requests my system has collected using locality sensitive hashing based on Moses Charikar's "Similarity ...
0
votes
2answers
91 views

For what type of problems nearest neighbor performs better

I'm trying to predict house prices. I use features like the area of the house, age of the house, etc. I turns out that knn (k-nearest neighbor) algorithm beats all the other powerful algorithms like ...
0
votes
0answers
123 views

comparing KNN with SVM on ROC chart

1) I wish to compare the performance of KNN and SVM on ROC chart. a)For KNN, I obtained a linear line on ROC Chart instead of curved one.From x=0 until x=0.275, y= 0.65 and from x=0.35 until x= 0.95, ...
1
vote
1answer
172 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
1
vote
2answers
108 views

kNN search using distance fitted to a training set of similar pairs

I want to perform k-nearest neighbor search in multidimensional space but not using for example L2 distance but I want user to specify some "similar" pairs-examples and then perform search using this ...
2
votes
1answer
137 views

Different use of neighbors in kNN classification algorithm

Suppose for each sample S in training set TS I already know the k nearest neighbors. The standard algorithms usually give S the same label as the majority of S's neighbors has. What other use of ...
1
vote
1answer
55 views

k-Nearest-Neighbor for Factorial Based Data

This week I read about using the kNN algorithm to predict the outcome of a continuous variable based on the input of one or more continuous variables. The example was predicting the price of a wine ...
1
vote
1answer
64 views

Distance independent approximation of Nearest Neighbor/k-NN.

Nearest neighbor/k-NN for use with Normalized Compression Distance. I wonder if there exist any approximation of NN/k-NN algorithm which work for all distance measures ? I would like to test ...
2
votes
1answer
162 views

k nearest neighbor with decision tree

A dataset has a few attributes. One of the attributes(attribute X) represents a distance with values expressed in meters. I use cross validation to estimate the performance of Decision tree and k ...
3
votes
1answer
287 views

Bayes decision boundary of Figure 2.5 in Elements of Statistical Learning

When I read "Elements of Statistical Learning", I met some difficulty in calculating the Bayes decision boundary of Figure 2.5. In the package ElemStatLearn, it ...
1
vote
2answers
166 views

Specifying the number of clusters in nearest neighbor clustering

I've got a distance matrix between examples. I want to cluster them into m clusters with a nearest neighbor algorithm which works like this: ...
0
votes
0answers
191 views

rknn - Error: too many ties in knn

I am trying to run rknn with rknn package but it appears an error I am not able to understand. Here's the code and the error: ...
2
votes
1answer
524 views

What is the optimal $k$ for the $k$ nearest neighbour classifier on the Iris dataset?

What is the optimal value of $k$, for an unweighted Euclidean kNN classifier applied to the Iris data set? Where optimal implies the value for $k$ which leads to the lowest generalisation error.
0
votes
2answers
192 views

Finding weights for variables in kNN

I'm using euclidean distance for kNN. I have labeled data, I have took logarithm of some variables to make them look more like normaly distributed and scaled them all. And now I would like to multiply ...
2
votes
1answer
112 views

Criteria to select the number of neighbors in the k-th-nearest-neighbor density estimation

I am doing a cluster analysis based on k-th-nearest-neighbor (KNN) method in SAS. The CLUSTER procedure requires to specify the $k$ (=number of neighbors to use for KNN density estimation) and I was ...
0
votes
0answers
197 views

SVM using RBF and nearest neighbor classification method

SVM using RBF kernel is claimed to be similar (equivalent) to the K nearest neighbor classification method. I am not very clear about the analysis process of building this kind of relationship. Thanks ...
2
votes
2answers
225 views

Dealing with lots of ties in kNN model

I have a large data set (400k rows X 60 columns) that I'm trying to use to build a knn model. I'm using the caret package version of ...
0
votes
0answers
85 views

Compare density estimate to true pdf

I'm trying to compare various density estimation methods. My dataset $D$ is generated from a fixed mixture of Gaussians (which allows me to estimate the true pdf $p(x)$). Then, I compute the estimated ...
1
vote
1answer
165 views

Using kknn regression on a time series

I'm trying to perform a knn regression on a sales time series using the kknn package. I can't see how you can restrict the neighbours to past sales only though. e.g. ...
2
votes
1answer
769 views

R knn variable selection

I have a data set that's 200k rows X 50 columns. I'm trying to use a knn model on it but there is huge variance in performance depending on which variables are ...
0
votes
0answers
125 views

LMNN: Simple Algorithm Description

B"H Hello, I understand the principles of LMNN (Large-Margin Nearest Neighbor) classification very well, but not all that well Weinberger's pseudo-code for its implementation. Can anyone simply (but ...
8
votes
3answers
2k views

How to plot decision boundary of a k-nearest neighbor classifier from Elements of Statistical Learning?

I want to generate the plot described in the book ElemStatLearn "The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Second Edition" by Trevor Hastie & Robert ...
9
votes
3answers
748 views

Combining machine learning models

I'm kind of new to datamining/machine learning/etc. and have been reading about a couple ways to combine multiple models and runs of the same model to improve predictions. My impression from ...
3
votes
1answer
62 views

Nearest neighbor where several vectors have exact same values

I'm trying to classify 6-dimension feature vectors in two different classes using the nearest neighbor algorithm. After analyzing my training data, I realized that the same vector is present numerous ...
1
vote
1answer
243 views

Principal component analysis before nearest neighbor search

I have a large data table (~500,000 rows) of normalized metrics (by Z-score) that looks like this: ...
3
votes
1answer
422 views

Text feature vector extraction

I have a class assignment to implement a couple existing ways to extract feature vectors from a given set of texts, so they can be used to classify those texts using k-nearest neighbour algorithm. The ...
1
vote
1answer
116 views

Sampling from kNN density estimate

Is there an easy/standard way of sampling from a density estimated by kNN? I've searched online but I can't seem to find how to do this. Any links or methods pointing me in the right direction would ...
1
vote
1answer
1k views

Text classification with RapidMiner

I have a folder for each class that contains some text files related to that class and need to implement a classifier for these files in RapidMiner with KNN algorithm. Is there any sample?
1
vote
0answers
78 views

How to determine the number of nearby samples for spatial estimation?

In many applications e.g, in mining engineering when we need to generate a map of dispersion of an element (e.g., copper) over the field of study, to depict depletion and concentration regions we have ...
1
vote
2answers
283 views

How can I implement multiclass k-NN?

I want to implement k-NN to use in a multi-class dataset. I found "A k-Nearest Neighbor Based Algorithm for Multi-label Classification" but didn't get the algorithm. Do you know any clear explanation ...
2
votes
2answers
307 views

Visualizing k-nearest neighbour?

Using R plot() and plotcp() methods, we can visualize linear regression model (lm) as an ...
3
votes
2answers
200 views

Accuracy of advanced parametric methods compared to kNN method

If kNN doesn't perform well for classification on a dataset, is there any hope for parametric methods to perform better? Kernel-based methods, SVM, random forests, and neural networks. Could any of ...
2
votes
1answer
422 views

Help understand kNN for multi-dimensional data

I understand the premise of kNN algorithm for spatial data. And I know I can extend that algorithm to be used on any continuous data variable (or nominal data with Hamming Distance). However, what ...
11
votes
2answers
767 views

When is “Nearest Neighbor” meaningful, today?

In 1999, Beyer et al. asked, When is "Nearest Neighbor" meaningful? Are there better ways of analyzing and visualizing the effect of distance flatness on NN search since 1999? Does [a given] data ...
7
votes
3answers
826 views

Adaptive kernel density estimators?

Can anyone report on their experience with an adaptive kernel density estimator? (There are many synonyms: adaptive | variable | variable-width, KDE | histogram | interpolator ...) Variable kernel ...

1 2