I have a dataset of dimensions 1500x200 where the predictors are both quantitative (discrete and continuous), as well as qualitative (categorical and ordinal) and the dependent variable is continuous. I've also performed standardization on these features.

I'm learning about PCA and visualization of PCA using biplots, and wanted to see if I could try it out on this dataset and see if I notice anything interesting. I found some code online that does this. About half way through I made a plot of the number of principal components versus explained variance. I expected that the first few principal components would explain much of the variance, but instead the plot looks sort of logarithmic where it takes over 30 principal components just to explain over 50% of the variance. Here's the plot:

enter image description here

So what I'm wondering is, is there something wrong methodologically with attempting to perform PCA on a dataset with mixed types such as the one I'm using? If so, is there a better visualization alternative than to create a biplot for this sort of data?

  • 2
    $\begingroup$ Are you using CATPCA? It is for mixed type variables. Second, it is nothing wrong with that 30+ components are needed to explain one third of the variance, it just means low correlations dominate in your data. $\endgroup$ – ttnphns Nov 29 '17 at 4:12
  • $\begingroup$ Thanks for your comment. So far I've just been using python sklearn's PCA implementation. If approximately half of my variables are continuous and half are dummy categorical variables, should I use CATPCA for all of them or somehow use PCA for the continuous half and CATPCA for the categorical half? $\endgroup$ – Austin Nov 29 '17 at 4:16
  • 1
    $\begingroup$ Dummy binary variables sets represent nomial variables. It is all right to use nominal variables in CatPCA, you don't have to recode them into the dummies, the program should do it internally. $\endgroup$ – ttnphns Nov 29 '17 at 5:01
  • $\begingroup$ Continuous vars okay to be mixed in also? $\endgroup$ – Austin Nov 29 '17 at 5:01
  • 1
    $\begingroup$ That depends on the implementation. Classic CatPCA algorthm is for discrete values only (scale can be interval, but values should not be fractional, continuous). That constraint is not of great inconvenience because you can bin continuous scale into, say, 30 or more subranges, represented by integer values 1, 2, 3,... With many subranges you don't distort your continuous data any greatly. $\endgroup$ – ttnphns Nov 29 '17 at 5:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.