Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [k-nearest-neighbour]

k-Nearest-Neighbor Classifiers These classifiers are memory-based, and require no model to be fit. Given a query point x0, we find the k training points x(r),r = 1,...,k closest in distance to x0, and then classify using majority vote among the k neighbors.

0
votes
0answers
3 views

k-Nearest Neighbors multiclass - what if we can't form the majority?

I'm watching a lecture on applying kNN to the digit recognition (MNIST dataset) problem. Let's say $n$ denotes the number of neighbors. This is a multiclass classification problem, so having an odd ...
0
votes
0answers
5 views

Find the number of misclassification [on hold]

I have a question in quiz what is the leave one out cross validation error for 1 knn and 3 knn on this dataset. the number of misclassifiactions.
0
votes
0answers
17 views

How does model accuracy compare across the folds in cross validation

If we have two cross validation from KNN and linear regression like this: KNN: ...
1
vote
1answer
22 views

Loss function for KNN Regressor

What is the Loss function for KNN Regressor? Would it be similar to OLS? If so what would be the main difference?
0
votes
1answer
19 views

Predict Click Through Rates for Google depending on the Position

I'm given the task to calculate expected click through rates (CTR) for rankings of a given site in Google, using the data Google provides in their Google Search Console. What I get there is basically ...
1
vote
2answers
24 views

Hypothetically, can we perfectly classify new instances given infinite support from validation and training? KNN

Hypothetically speaking, Given infinite amounts of training data and validation data, can we achieve perfect classification (score of 1) given ML algorithms such as KNN? Thank you.
1
vote
1answer
19 views

Dimensionality Reduction for Optimally Preserving KNN

Do any dimensionality reduction techniques find embeddings which optimally preserve the K-nearest neighbors of each point? If no algorithm provably does this, are there algorithms which heuristically ...
0
votes
1answer
20 views

Training data grow with K-NN and Naive Bayes

My doubt is about the grow of the training data using K-NNs and Naive Bayes. As it grows larger, does prediction (on test data) become also computationally harder?
0
votes
0answers
42 views

Is this an example of overfitting?

I am trying to predict some future values using either KNN or regression model. I have about 9 independent variables that do not seem to have strong correlation to each other (Not completely sure ...
0
votes
1answer
27 views

A simple concept question regarding Bootstrapping

I am having a difficulty understanding whether I can use the bootstrapping method for prediction. First off, my data is as follows. where personal Income is the dependent variable (y), and GPA, Age, ...
0
votes
0answers
13 views

k nearest neighbor for likert data

I have a data set which contains several variables. All these variables are catgorical variables. I want to use k nearst neighbour method. When I am using this technique do I need to convert the data ...
0
votes
0answers
19 views

How to evaluate the goodness of a KNN-based recommender system?

I'm building a content-based recommender system and I'm using KNN. That means, for each test instance I'm using KNN to find the most similar objects, and then come up with a recommendation. However I'...
1
vote
1answer
31 views

Propensity score matching: bias adjustment

I'm using propensity score matching to match similar individuals. I.e., I first estimate a propensity score (the probability of treatment conditional on some set of variables) and then match on the ...
3
votes
1answer
26 views

Quantifying a manifold folding unto itself

I have a dataset of ~7k scattered points in 3D which represents a manifold that may or may not "fold unto itself". Here's an example where this does happen (look at the top-right yellow triangles): ...
1
vote
0answers
34 views

What if the best k in k-NN is equal to the number of data points?

I need to train a regressor (in the machine learning sense). I have tried many different methods and so far nothing works better than just a constant prediction. In other words in looked like I have ...
0
votes
0answers
16 views

Citation for selection of weighted KNN weights via cross validation

Examples of choosing the hyperparameter $K$ for the $K$-nearest neighbors algorithm via cross-validation abound in the literature. As with most `local' methods, a relatively trivial generalization of ...
0
votes
0answers
16 views

Probability density function of a data point given the locations of its four nearest neighbors

The goal is to find the probability density function $p(\mathbf{x} | \mathbf{c}_1,\mathbf{c}_2,\mathbf{c}_3,\mathbf{c}_4)$. Here $\mathbf{x},\mathbf{c}_i \in \mathbb{R}^d$. $\mathbf{c}_1,\mathbf{c}_2,\...
0
votes
0answers
27 views

How detect outliers with get.knn

I am trying to detect outliers in a multivariate data with R using get.knn() function from the ...
2
votes
1answer
41 views

K-NN optimal value of 'K'

How can I find the optimal value of 'K' in K-NN using the cross_val_score function, with scoring metric as auc_score? Do I need ...
0
votes
1answer
37 views

KNN regression: Why does my In sample RMSE look like my out of sample RMSE across K values?

I'm expecting the RMSE plot for my KNN regression model to look like the above image but I'm getting the below when running my code hosted here. Any ideas on what could cause this? I believe something ...
0
votes
0answers
30 views

Some K-nearest neighbour predictions do not make sense

Main problem: Why are the predicted rent of customers with a certain level of income higher than rent of customers with both higher and lower levels of income keeping other variables fixed? And how ...
0
votes
0answers
23 views

Predicting response on boundaries of observations with KNN

Suppose a bivariate process is observed $(X_1, Y_1), \ldots, (X_n, Y_n)$ that follows the probability model of ordinary linear regression $$Y_i = \alpha + \beta X_i + \epsilon_i, $$ and assume $\...
1
vote
0answers
31 views

Naive Bayes and kNN accuracy

Assume that a large number of binary features are added to a dataset with two class labels c1 and c2, such that for each added feature f, the class conditional probability P(f = 0|c1) = P(f = 0|c2). ...
1
vote
1answer
189 views

how to prevent overfitting with knn

Using too low a value of K gives over fitting. But how is overfitting prevented: How do we make sure K is not too low And are there any other precautions taken in k-nn that help prevent over ...
0
votes
0answers
7 views

For ADASYN, if the neighbourhood of a minority sample contains no other minority sample, do I double the sample?

In ADASYN, for the last step in the paper linked below, if there exists no other minority class in the k-NN other than the one minority example, do we simply just double the training example? Because ...
0
votes
1answer
26 views

Addressing “Unbalanced Features” or Feature Taxonomy for Nearest Neighbor / Similarity Calculations

The main question is how to address an imbalance in representation of feature "sets" when calculating similarity. I'll motivate with an example scenario: Suppose we have objects described by a binary ...
0
votes
1answer
27 views

What are characteristics of “local methods” in data science?

And what is the opposite? "Global" methods? What the differences are? I have found reference to "local" methods in this answer about KNN: https://stats.stackexchange.com/a/104261/107213
2
votes
1answer
92 views

How to get model in knn()?

Given I have classified my inputs using R's built-in knn(): ...
1
vote
0answers
44 views

t-SNE for finding nearest neighbors

I had a question about dimensionality reduction for finding nearest neighbors and was hoping someone could help me out here. Suppose I have good features for images, say penultimate layer features ...
0
votes
0answers
448 views

K Fold Cross Validation in in KNN algorithm

I have a dataset with 2000, observations and 21 columns. Using KNN, I want to classify validating data using last column price_range which contains the ...
1
vote
1answer
82 views

How does scikit-learn's kNN model handle zero-distances when using inverse distance weighting?

OK, I could go through the code to figure this out but I feel something Googleable doesn't hurt. When I'm using a kNN classifier with (inverse) distance weighting, how does it handle cases whereby ...
1
vote
1answer
85 views

Why effective number of parameters in K nearest neighbor is N/k?

Bellow is my deduction: According to the definition of k-NN fit, we have $$\hat{Y}(x) = \frac{1}{k} \sum_{x_i \in N_k(x)}^{N}= \frac{1}{k}diag(a_1, a_2,..., a_N)y$$ where $N_k(x)$ is the neighborhood ...
0
votes
0answers
20 views

Smoother Regression Results with KNN in Regression

Weak learners like tree, knn etc. provides regression like stairs. You can examine this example: https://scikit-learn.org/stable/auto_examples/neighbors/plot_regression.html I want to make a ...
4
votes
1answer
51 views

KNN parameter tuning with cross validation: score draw

I'm trying to use the KNN method for binary classification. When trying to find the best 'k' parameter (the amount of neighbours that the algorithm looks at) I train a model on my training set and ...
3
votes
1answer
90 views

The violation of triangle inequality in KNN

If the 0<p<1 in the distance metrics, then the triangle inequality is violated. The question as follows Does the violation of this inequality affect the ...
1
vote
0answers
147 views

Voronoi diagrams for kNN

This question is obviously related to Are Voronoi diagrams used in kNN algo implementations?, but I'd like to ask a couple more question than there. I am considering using Voronoi diagram to speed up ...
0
votes
0answers
8 views

Learn mapping function item to item

I'm trying to resolve a problem in which a set of items have to be mapped to another set of items. The mapping has to be one to one and the items have attributes that describe them (e.g. name, ...
1
vote
0answers
42 views

Calculating log-likelihood or BIC of KNN and random forest/boosting models

I am trying to find the BIC of the KNN, random forest and boosting models (for regression, not classification) to use in a combined model that uses Bayesian model averaging to predict a target ...
1
vote
0answers
30 views

K nearest neighbour estimation example

I want to know how the k–Nearest-Neighbor Estimation work. for example i want to classify test sample (0.5, 0) in Unbiased K Nearest Neighbor approximation with k = 3, and Euclidean distance: $d(𝑥,𝑦)...
3
votes
0answers
68 views

Imputation and nested cross-validation

I am planning to do a nested cross-validation analysis using regularized regression. The inner loop will be used for model tuning and the outer loop for model assessment (test set). Because some data ...
1
vote
0answers
30 views

Cross Validation, Variable Definition [closed]

I am working on a 10-fold cross validation problem, and am having an issue with part of my code. Specifically, I'm having a problem with my "for (1 in nfold)" argument, and with the variable length of ...
1
vote
1answer
534 views

Knn Decision boundary

I am new to machine learning and trying to draw decision boundary for k nearest neighbor where k=3. I know that the decision boundary for k=1 would be the perpendicular bisector between two different ...
0
votes
0answers
47 views

can I fill missing values by using target variable?

I have a 3 column data with 2 features and 1 target variable. But the first features (numeric) have a large number of missing values. If I use kNN to fill in the missing values, I am wondering can I ...
2
votes
1answer
286 views

Other distances than euclidean distance in knn [closed]

Suppose I want to fit a k-nearest-neighbour using caret package in R: ...
4
votes
1answer
238 views

Does class balancing introduce bias?

I have a data set that is imbalanced, the prediction rate is not much better than the base line without doing any class balance. I have two classes and I can't collect more data. What I have done: ...
2
votes
2answers
247 views

Does the presence of the outliers affect the 1NN algorithm?

I am working on KNN algorithm. I uploaded and prepared the following dataset. ...
0
votes
1answer
26 views

Similarity between 2 profiles (observations). Is it possible to generate a % similarity?

I have multiple profiles for 10 different people. Each person has been measured for 5 different continuous variables of different magnitudes. So my dataframe is 10x5 where each row represents a person ...
4
votes
2answers
4k views

Training error in KNN classifier when K=1

I got this question in a quiz, it asked what will be the training error for a KNN classifier when K=1. What does training mean for a KNN classifier? My understanding about the KNN classifier was that ...
1
vote
1answer
20 views

Lazy Learning given Submission Files in Competitions

In machine learning settings where you get the "real data" beforehand (e.g. a submission file in competitions like Kaggle), models can be "build around the data" that actually has to be predicted. ...
-1
votes
1answer
23 views

K-means in R: complete case analysis followed by nearest-neighbor assignment for partial data

I have a dataset of 3K observations with only 162 being a complete case. I have read here that it is possible to run knn on the complete cases and then conduct a nearest neighbour assignment for ...