4
$\begingroup$

I am working with a dataset of around 4000 variables. I decided to carry out a principal component analysis (PCA) for the data, but I am not quite sure about the suitable number of variables I should include in the test.

Would feeding a big number (such as 4000) variables interfere with the PCA accuracy? As far as I can understand from the definition and the way the PCA is conducted, the number of variables should not matter, but I am not 100% sure and I couldn't find any source talking about the effect of number of variables on PCA.

My question in simple terms would be: should I just include all the variables I have into the PCA test or should I do other tests to reduce the number of variables a little before doing the PCA?

$\endgroup$
  • $\begingroup$ How do you define "PCA accuracy"? One of the main applications of PCA is to reduce the dimensionality when there are many variables. So yes, you should use all of the variables. $\endgroup$ – January Sep 20 '13 at 12:40
  • $\begingroup$ Hi @January, I was concerned that the "importance" or the relationship between the principal components and the variables might change when huge number of variables are included. I have applied PCA before for a relatively small number (100) variables, but wasn't sure if I can still trust the method with this large number of variables. Thanks for your reply :) Can you rewrite it as an answer please? $\endgroup$ – Error404 Sep 20 '13 at 13:28
  • $\begingroup$ Certainly the inclusion of a different subset of variables is liable to change the nature of the obtained components and the patterns of relationships within and among components. If you think about it, PCA would be a strange and pretty useless beast otherwise. $\endgroup$ – rolando2 Jan 23 '15 at 0:36
6
$\begingroup$

One of the main applications of PCA is to reduce the dimensionality when there are many variables. So yes, you should use all of the variables. I routinely apply PCA to tens of thousands of variables (gene expression data) and it works very well.

What can happen is that when analysing PCA you will have to look into more than the first two or three components. Often, the factor that you would like to understand does not contribute to the majority of variance and for example you will see nice clustering of your samples in the fifth or tenth component (yes, I have seen cases like this).

$\endgroup$
  • $\begingroup$ interesting, I wasn't considering a clustering test for my data, I will give it a go. Many thanks. $\endgroup$ – Error404 Sep 20 '13 at 14:48
  • $\begingroup$ I am also applying PCA to my financial dataset where I have more than 4000 variables (and 9500 observations). The groupings of original variables in PCs are meaningful but their loadings are very low (around 0.01). And there are many highly correlated original variables (more than 0.8) in the dataset. Can I still use principal components with such very low loadings? $\endgroup$ – mlee_jordan Dec 10 '17 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.