I have encountered the presentation and one of the ideas mentioned there is as follows. Suppose, that there is a sample of objects with 100 features, only 5 of which are informative. On the 5th slide it recommends to extract ~20 of these features by PCA, as the informative features are likely to keep the desired 5.

Can anyone explain the nature of this recommendation? Why don't we extract exactly 5 features by PCA? Or at least 10, as intuitively 20 seems to be a bit overkill.

  • 2
    $\begingroup$ Keep in mind that none of the PCA features map directly to your original features in a 1:1 manner. Each PCA feature is a combination of your original features. You'll almost certainly need more than 5 PCA features to capture all of the variance in your original important features. $\endgroup$ – Nuclear Wang Apr 20 '16 at 14:38

I think that this one is just a rough estimate as the following lines states that keeping 20 axes is a lot better than having the 100 initial variables and not that far from 5 (if you kept only the 'good' initial variables). 20 is high enough to ensure capturing the 5 but low enough so that PCA is still a pertinent approach.

What I usually do to decide on the number of dimensions to keep is just according to the percentage of inertia I want to keep (95% often times).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.