3
$\begingroup$

I'm new to PCA and I'm trying to apply it to a dataset I have with 15 different features. I normalized my dataset before applying PCA and used the PCA method in the decomposition function from sklearn. I was hoping that a few PCAs (probably less than 10) would be able to explain 90% of the variance of the data matrix. But the cumulative variance I got is

 [0.21  0.323 0.413 0.486 0.555 0.619 0.681 0.74  0.794 0.844 0.89  0.922
 0.953 0.981 0.9998]

which means that I need at least 12 PCs to explain 90% of the variance and 15 PCs to explain 100%. So it seems like PCA does not reduce the dimensionality of my dataset. Does it mean that the 15 features I have are not redundant? like there is no redundancy in my dataset and it's better not to eliminate any of the 15 features I currently have?

Below is my features correlation heatmap. enter image description here

$\endgroup$
4
  • $\begingroup$ Hard to say withouth looking to all factor loadings. Perhaps two or three variables are highly correlated. You could check a correlation matrix of your variables to see if it is the case: towardsdatascience.com/… $\endgroup$
    – LuizZ
    Nov 20 '20 at 1:35
  • $\begingroup$ Thanks for the input! I added the correlation heatmap. $\endgroup$
    – JY078
    Nov 20 '20 at 12:50
  • 3
    $\begingroup$ Most of those correlations are quite small in magnitude so your PCA results are not surprising. $\endgroup$
    – mdewey
    Nov 20 '20 at 13:05
  • 1
    $\begingroup$ Take it to the extreme and consider what PCA does when your observed variables are uncorrelated (so a diagonal (empirical) covariance matrix). $\endgroup$
    – Dave
    Nov 20 '20 at 13:09
2
$\begingroup$

Yes. Because you have a fairly small amount of features already (15), it makes sense if you weren't able to reduce the dimensionality much further without reducing the explanation for variance. PCA is often done on datasets with hundreds or thousands of features to reduce the dataset. Although, to note, if you did have highly-redundant features, it is also totally possible that you could have reduced your dataset down to fewer features.

Yes, you are right that there is not much redundancy in your dataset and you shouldn't reduce the features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.