I am working on a dataset with with 100 or so X variables and the effects on a single Y.

As with previous threads this contains a lot of fairly significant correlations. I considered using PCA but did not really want to loose the original variables by creating linear combinations (factors).
X variables also in this case have some nasty distributions and some are categorical. However, my Y is nicely normal.

With this I decided to use bootstrapping with partition analysis.

Great, a few predictors were produced that all seemed to add to the explanation of Y, however on further investigation it is looking as if only one or 2 predictors are responsible for the Y variation. Partial correlation appears to be the reason why others are included.

Indeed, when run together in a linear model VIFs are below 10 but many are around 5 Now putting these variables into a PCA produces suspicious results: PCA appears to pull 3 factors of worth with loadings of these.

Rotated factor loadings

Factor 1        Factor 2        Factor 3
0.5710189449    0.5594104391    0.0092385097
0.0573348956    0.128427451    0.8688140622
-0.029701697    0.8398533899    0.3350420004
-0.024820744    -0.931953121    0.1360273344
0.434265868      0.8133639053   0.022057529
0.8103764005    0.3739830113    -0.225363624
0.4622020154    0.2581755128    -0.208011559
0.8560481143    0.3198727429    -0.191875778
0.7308835871    -0.08016601     0.2661637317
0.8815834293    0.2432798016    -0.120213308
0.8507444962    -0.044814927    0.3006428831
-0.857600499    -0.205246822    -0.036006619
0.6900628882    0.1578752523    0.0377927821
0.7131252931    -0.289263321    0.3166515143

Is this effective saying that if I do not want to use recombined factors I only really have a couple of variables?
I am unsure if I can use PCA this way.

  • $\begingroup$ What did you want the PCA to do? $\endgroup$ – Peter Flom Jan 10 '14 at 11:44
  • $\begingroup$ i wanted to use PCA to identify variables that are correlated but i do not want to put the factors into a model, i require the variables to be in their original form. when i originally tried to use PCA on the complete dataset i couldnt achieve this due to what i thought was so much mild to severe correlation $\endgroup$ – Samuel Jan 10 '14 at 11:47
  • 1
    $\begingroup$ A better method is to use condition indices to diagnose collinearity, or correlation matrix to see which variables are correlated. I am not sure what PCA does for you here. $\endgroup$ – Peter Flom Jan 10 '14 at 11:49
  • $\begingroup$ the correlation matrix indicates correlation within the variables but i am always unsure at what level to actually determine this to be significant $\endgroup$ – Samuel Jan 10 '14 at 11:51
  • $\begingroup$ That's where condition indices are very useful. The output also indicates which variables contribute to high condition indices. $\endgroup$ – Peter Flom Jan 10 '14 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.