I have a dataset which originally had 34000 features but only 4968 samples of data. To which I applied PCA and derived 4968 principal components.
Here is a table detailing the principal components derived and their respective cumulative explained variance
So my question is this. Why do models start to perform extremely poor when i choose K components which amount close to or equal to 100% explained variance?
Some example results I have logged
- Linear SVM
- Logistic Regression