I am doing KNN classification using PCA method. For this, I first did PCA on train data and then predicted components for test set using train PCA. So my train PCA plot looks like this:
I decided to go for first 15 components for test set prediction on KNN classifier. Based on this, I got the following error estimates (plot below). The accuracy estimate (~30%) is acceptable in this case because the dataset has multidimensional responses (i.e response A,B and C grouped together). The error estimate from KNN looks like this:
So the questions I have are:
- I have 99 percent variability explained by 15 component and I decided to select 15 components for KNN training. Is there a minimum/maximum variability I should consider for this?
- What's the disadvantage of selecting too many or too few components? Can you not select all components?