Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [k-nearest-neighbour]

k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

0
votes
0answers
25 views

Some K-nearest neighbour predictions do not make sense

Main problem: Why are the predicted rent of customers with a certain level of income higher than rent of customers with both higher and lower levels of income keeping other variables fixed? And how ...
0
votes
0answers
20 views

Predicting response on boundaries of observations with KNN

Suppose a bivariate process is observed $(X_1, Y_1), \ldots, (X_n, Y_n)$ that follows the probability model of ordinary linear regression $$Y_i = \alpha + \beta X_i + \epsilon_i, $$ and assume $\...
1
vote
0answers
23 views

Naive Bayes and kNN accuracy

Assume that a large number of binary features are added to a dataset with two class labels c1 and c2, such that for each added feature f, the class conditional probability P(f = 0|c1) = P(f = 0|c2). ...
1
vote
1answer
39 views

how to prevent overfitting with knn

Using too low a value of K gives over fitting. But how is overfitting prevented: How do we make sure K is not too low And are there any other precautions taken in k-nn that help prevent over ...
0
votes
0answers
6 views

For ADASYN, if the neighbourhood of a minority sample contains no other minority sample, do I double the sample?

In ADASYN, for the last step in the paper linked below, if there exists no other minority class in the k-NN other than the one minority example, do we simply just double the training example? Because ...
0
votes
1answer
16 views

Addressing “Unbalanced Features” or Feature Taxonomy for Nearest Neighbor / Similarity Calculations

The main question is how to address an imbalance in representation of feature "sets" when calculating similarity. I'll motivate with an example scenario: Suppose we have objects described by a binary ...
0
votes
1answer
24 views

What are characteristics of “local methods” in data science?

And what is the opposite? "Global" methods? What the differences are? I have found reference to "local" methods in this answer about KNN: https://stats.stackexchange.com/a/104261/107213
2
votes
1answer
26 views

How to get model in knn()?

Given I have classified my inputs using R's built-in knn(): ...
1
vote
0answers
27 views

t-SNE for finding nearest neighbors

I had a question about dimensionality reduction for finding nearest neighbors and was hoping someone could help me out here. Suppose I have good features for images, say penultimate layer features ...
-1
votes
0answers
21 views

Standardize/Normalize data combined with numerical, categorical, scale, and binary attributes for kNN

I have data consisting of continuous, categorical, scale (1-10), and binary attributes. How do I normalize all of this data to be used for k-Nearest Neighbors? I understand you have to normalize the ...
0
votes
0answers
118 views

K Fold Cross Validation in in KNN algorithm

I have a dataset with 2000, observations and 21 columns. Using KNN, I want to classify validating data using last column price_range which contains the ...
1
vote
1answer
22 views

How does scikit-learn's kNN model handle zero-distances when using inverse distance weighting?

OK, I could go through the code to figure this out but I feel something Googleable doesn't hurt. When I'm using a kNN classifier with (inverse) distance weighting, how does it handle cases whereby ...
1
vote
1answer
39 views

Why effective number of parameters in K nearest neighbor is N/k?

Bellow is my deduction: According to the definition of k-NN fit, we have $$\hat{Y}(x) = \frac{1}{k} \sum_{x_i \in N_k(x)}^{N}= \frac{1}{k}diag(a_1, a_2,..., a_N)y$$ where $N_k(x)$ is the neighborhood ...
0
votes
0answers
16 views

Smoother Regression Results with KNN in Regression

Weak learners like tree, knn etc. provides regression like stairs. You can examine this example: https://scikit-learn.org/stable/auto_examples/neighbors/plot_regression.html I want to make a ...
4
votes
1answer
27 views

KNN parameter tuning with cross validation: score draw

I'm trying to use the KNN method for binary classification. When trying to find the best 'k' parameter (the amount of neighbours that the algorithm looks at) I train a model on my training set and ...
3
votes
1answer
68 views

The violation of triangle inequality in KNN

If the 0<p<1 in the distance metrics, then the triangle inequality is violated. The question as follows Does the violation of this inequality affect the ...
1
vote
0answers
28 views

Voronoi diagrams for kNN

This question is obviously related to Are Voronoi diagrams used in kNN algo implementations?, but I'd like to ask a couple more question than there. I am considering using Voronoi diagram to speed up ...
0
votes
0answers
8 views

Learn mapping function item to item

I'm trying to resolve a problem in which a set of items have to be mapped to another set of items. The mapping has to be one to one and the items have attributes that describe them (e.g. name, ...
0
votes
0answers
27 views

Calculating log-likelihood or BIC of KNN and random forest/boosting models

I am trying to find the BIC of the KNN, random forest and boosting models (for regression, not classification) to use in a combined model that uses Bayesian model averaging to predict a target ...
1
vote
0answers
29 views

K nearest neighbour estimation example

I want to know how the k–Nearest-Neighbor Estimation work. for example i want to classify test sample (0.5, 0) in Unbiased K Nearest Neighbor approximation with k = 3, and Euclidean distance: $d(𝑥,𝑦)...
3
votes
0answers
51 views

Imputation and nested cross-validation

I am planning to do a nested cross-validation analysis using regularized regression. The inner loop will be used for model tuning and the outer loop for model assessment (test set). Because some data ...
1
vote
0answers
28 views

Cross Validation, Variable Definition [closed]

I am working on a 10-fold cross validation problem, and am having an issue with part of my code. Specifically, I'm having a problem with my "for (1 in nfold)" argument, and with the variable length of ...
0
votes
1answer
236 views

Knn Decision boundary

I am new to machine learning and trying to draw decision boundary for k nearest neighbor where k=3. I know that the decision boundary for k=1 would be the perpendicular bisector between two different ...
0
votes
0answers
26 views

can I fill missing values by using target variable?

I have a 3 column data with 2 features and 1 target variable. But the first features (numeric) have a large number of missing values. If I use kNN to fill in the missing values, I am wondering can I ...
1
vote
1answer
162 views

Other distances than euclidean distance in knn [closed]

Suppose I want to fit a k-nearest-neighbour using caret package in R: ...
4
votes
1answer
132 views

Does class balancing introduce bias?

I have a data set that is imbalanced, the prediction rate is not much better than the base line without doing any class balance. I have two classes and I can't collect more data. What I have done: ...
2
votes
2answers
151 views

Does the presence of the outliers affect the 1NN algorithm?

I am working on KNN algorithm. I uploaded and prepared the following dataset. ...
0
votes
1answer
22 views

Similarity between 2 profiles (observations). Is it possible to generate a % similarity?

I have multiple profiles for 10 different people. Each person has been measured for 5 different continuous variables of different magnitudes. So my dataframe is 10x5 where each row represents a person ...
2
votes
2answers
2k views

Training error in KNN classifier when K=1

I got this question in a quiz, it asked what will be the training error for a KNN classifier when K=1. What does training mean for a KNN classifier? My understanding about the KNN classifier was that ...
1
vote
1answer
20 views

Lazy Learning given Submission Files in Competitions

In machine learning settings where you get the "real data" beforehand (e.g. a submission file in competitions like Kaggle), models can be "build around the data" that actually has to be predicted. ...
-1
votes
1answer
20 views

K-means in R: complete case analysis followed by nearest-neighbor assignment for partial data

I have a dataset of 3K observations with only 162 being a complete case. I have read here that it is possible to run knn on the complete cases and then conduct a nearest neighbour assignment for ...
0
votes
0answers
37 views

Why does my kNN model achieve perfect performance on my test set?

I have trained a KNN model using cross validation to minimize a custom loss function. The model clearly overfits the training data, achieving a loss of zero but exhibits variation in 10-fold cross ...
0
votes
0answers
29 views

Intuitive sense of K-nearest neighbor mutual information estimation

I am using the R package Parmigene to estimate the mutual information (MI) between different proteins. The data is spectral counts, which is nonnegative and mostly zero. I want to know what is ...
0
votes
1answer
94 views

Regression kNN model vs. Classification kNN model

I was wondering what is the difference between regression kNN model and classification kNN model. I tried Googling and no success. In presentation from lectures we only have graphs of errors of ...
8
votes
1answer
166 views

Which type of data normalizing should be used with KNN?

I know that there is more than two type of normalizing. For example, 1- Transforming data using a z-score or t-score. This is usually called standardization. 2- Rescaling data to have values ...
5
votes
2answers
200 views

Regression as a way to determine variable importance

At my work, we employ a nearest neighbor algorithm to classify records. Part of this process, of course, includes determining which features to use as auxiliary information in the algorithm. Also, ...
0
votes
1answer
142 views

Can you derive variable importance from a nearest neighbor algorithm?

While the helpful tooltip warned me this question was subjective, I don't think it is. It should be fairly objective to state from a theoretical perspective whether or not you can establish the ...
0
votes
0answers
24 views

Prediction with categorical and continuous Variables

I want to predict the result of a match in a video game (win or loose). It's 5 players against 5 players game, who each plays a specific character. I have : the ID of each character (there are 150 ...
1
vote
0answers
14 views

KNN with categorical values can not predict correctly

I am trying to build a model that given an item, predicts which store it belongs to. I have a data-set of ~250 records which are supposed to be items in different online stores. Each record is ...
0
votes
0answers
33 views

How can I find the k-nearest neighbors for a collection of linear time series data?

I need to figure out how to determine the nearest neighbors of an "optimal" line, as illustrated in a simplified figure, linked below. The blue, orange, green, and purple lines represent the best fit ...
3
votes
1answer
205 views

How to overcome the computational cost of the KNN algorithm?

How to overcome the computational cost of the KNN algorithm when the dataset is large. Bear in mind that I am using my own function for building the knn algorithm in R Is there any manipulation to ...
1
vote
1answer
214 views

Is it possible to compare two different datasets with a different range of values?

I am currently working with the k-Nearest Neighbors (KNN) algorithm. Is it possible to compare two datasets with different ranges of values? In other words, I have one dataset with a range $\in [0,1]...
0
votes
0answers
36 views

kNN prediction using weighted nearest neighbor examples

I've been struggling with this question for about an hour and I can't seem to wrap my head around it. Can someone explain how to use the nearest neighbor examples to make a prediction? This is going ...
1
vote
0answers
43 views

How to select value of k in K-NN when using Transfer Entropy?

I was reading about Transfer Entropy and I came across the estimators used to calculate TE, one of them being the Kraskov Estimator: $ T_{X \rightarrow Y} = \frac{p(Y_{n+1}, Y(k)_{n}, X(l)_{n})*log(...
0
votes
0answers
16 views

KNN: scale data with a concave function to give lower numbers more “weight”?

I am scaling (normalizing) a particular feature (column) in my K-Nearest Neighbors model using the classic min-max (0 to 1) technique. I know I could give the feature more importance than other ...
3
votes
1answer
195 views

Why is N/k the effective number of parameters in k-NN?

For the sake of completeness, $k$-nearest neighbor method classifies a point in space by comparing the average over the labels of $k$ nearest neighbors with $0.5$. The book Elements of Statistical ...
0
votes
0answers
56 views

Nearest Neighbors without fixed dimensions

I got following situation: In a gym there are m trainings-spots (for example a circuit or different trainings-tools). Each trainings-spot has his own unique number to identify it. People training in ...
2
votes
2answers
625 views

Lack of understanding of LOOCV

I am trying to utilize LOOCV in the data partition in R. The idea of LOOCV is to train the model on n-1 set and test the model on the only remaining one set. Then, is to repeat this process n times ...
0
votes
1answer
130 views

Why is my model so accurate when using knn(), where k=1?

I am currently using genomic expression levels, age, and smoking intensity levels to predict the number of days Lung Cancer Patients have to live. I have a small amount of data; 173 patients and 20,...
1
vote
1answer
83 views

Do I need data separation in KNN?

I am trying to use KNN with cancer data. I have split my data into train and test sets. Am I right or I should use LOOCV?