Higher accuracy in training set than test set in KNN classifier If I got an accuracy of 95% in the training set, and 80% accuracy in the test set, what could be the explanation for that? it sounds like a pretty basic question, but I just can't put it into words correctly.
It's supposed to be like that, I know that far. I interpret it like this:
" There is overfitting in the KNN model, the model learned the training data too well, and now it uses the features it learned in the training data in a false way. The model is familiar with the training data more, after all, it was trained on it, and the test data it has never seen before so it's only natural to get these results ".
What do you think of this explanation?
 A: If you use too small values for k (e.g. k=1) you get decision boundaries which are very erratic and that leads to overfitting.
It means that if k is too small the decision boundaries depend highly on the sample that you have. If you had another sample (by chance) the decision boundaries would look very different.
Below are two pictures from the book "The Elements of Statistical Learning Data Mining, Inference, and Prediction" by Hastie, Tibshirani and Friedman illustrating this. Choosing larger k leads to more smooth decision boundaries and less variance in the model.
Whether a gap of 95% to 80% is reasonable depends on the sample size and the dimensionality of your data set I would say. In general though the gap will widen for smaller k and get smaller for larger k, but never disappear.


A: The explanation for this situation: your testing data contains features which are not in the training data (so the model see the new features as unknown and make wrong prediction)
You can use one (or both) of the methods bellow:
1-Split the data again but with random_state >80 according to your dataset
2-Resize the training and the testing set
if you have a large data you must use another algorithm KNN is bad for large data
