Skip to main content
2 of 4
deleted 1 character in body
MAPK
  • 283
  • 3
  • 11

How to decide on optimum number of components for KNN classification

I am doing KNN classification using PCA method. For this, I first did PCA on train data and then predicted components for test set using train PCA. So my train PCA plot looks like this:

PCA variance plot

I decided to go for first 15 component for test set prediction on KNN classifier. Based on this I got the following error estimates. The accuracy estimate (~30%) is acceptable in this case because the dataset has multidimensional responses (i.e response A,B and C grouped together). The error estimate from KNN looks like this:

Error estimate of KNN classifier

So the questions I have are:

  1. I have 99 percent variability explained by 15 component and I decided to select 15 components for KNN training. Is there a minimum/maximum variability I should consider for this?
  2. What's the disadvantage of selecting too many or too few components? Can you not select all components?
MAPK
  • 283
  • 3
  • 11