2
$\begingroup$

There are 2 sides to this question. On the first side, we are trying to run PCA.

While running PCA on a dataset having 62 independent variables (IVs), I have found that first component (PC1) explains ~90% of variance and that of variables or parameters contributing to this component, a variable say V1 has highest contribution. This means that V1 plays a big role in capturing major chunk of variance in the whole dataset. And hence, this V1 is important.

On the second side, we have found correlations of all IVs with the dependent variable (DV) Y, so that we can drop IVs with the correlation coefficients (with Y) less than a particular threshold, say 0.2. This exercise is done so that we have less number of IVs for building a model. Here, we find that if we consider this threshold, we have to drop V1.

This to me is surprising because V1 is very important when you look at it from PCA point of view but is very less important when you look at it from point of view of explaining DV.

Looking at these 2 sides, should we drop V1 now and why? Please share your inputs. If you need any other input from my end, please do ask.

PS - we are doing this exercise in R.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ You seem to equate "contribution" with loading. Here come preliminary remarks. First, one should be aware that loading is not eigenvector's element. Second, loading is the weight of dependency of a variable on a PC, not vice versa. $\endgroup$
    – ttnphns
    Commented Sep 1, 2016 at 8:33
  • $\begingroup$ Related: stats.stackexchange.com/questions/141864 $\endgroup$
    – amoeba
    Commented Sep 1, 2016 at 12:41
  • $\begingroup$ Thank you every one for replying. The links that you have shared are really useful. $\endgroup$
    – skumar
    Commented Sep 1, 2016 at 16:27

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.