I am working on a dataset with with 100 or so X variables and the effects on a single Y.
As with previous threads this contains a lot of fairly significant correlations. I considered using PCA but did not really want to loose the original variables by creating linear combinations (factors).
X variables also in this case have some nasty distributions and some are categorical. However, my Y is nicely normal.
With this I decided to use bootstrapping with partition analysis.
Great, a few predictors were produced that all seemed to add to the explanation of Y, however on further investigation it is looking as if only one or 2 predictors are responsible for the Y variation. Partial correlation appears to be the reason why others are included.
Indeed, when run together in a linear model VIFs are below 10 but many are around 5 Now putting these variables into a PCA produces suspicious results: PCA appears to pull 3 factors of worth with loadings of these.
Rotated factor loadings
Factor 1 Factor 2 Factor 3 0.5710189449 0.5594104391 0.0092385097 0.0573348956 0.128427451 0.8688140622 -0.029701697 0.8398533899 0.3350420004 -0.024820744 -0.931953121 0.1360273344 0.434265868 0.8133639053 0.022057529 0.8103764005 0.3739830113 -0.225363624 0.4622020154 0.2581755128 -0.208011559 0.8560481143 0.3198727429 -0.191875778 0.7308835871 -0.08016601 0.2661637317 0.8815834293 0.2432798016 -0.120213308 0.8507444962 -0.044814927 0.3006428831 -0.857600499 -0.205246822 -0.036006619 0.6900628882 0.1578752523 0.0377927821 0.7131252931 -0.289263321 0.3166515143
Is this effective saying that if I do not want to use recombined factors I only really have a couple of variables?
I am unsure if I can use PCA this way.