I have a question regarding principal component regression (regression of a DV on principal components). I have 4 components in my PCR and the third component is non-significant as a predictor. What does that mean? It's curious that the "weaker" fourth component however is significant. Can anyone explain this to me?
$\begingroup$
$\endgroup$
5
-
2$\begingroup$ Please explain what information you are using to determine that components are "significant" or not: this is not a standard part of PCA. $\endgroup$– whuber ♦Dec 17, 2013 at 23:30
-
1$\begingroup$ I have 20 variables which are my predictors and get through the PCA my 4 components. Then I do a PCR where I look where I have my 4 components as predictors and a further variable as dependent variable. The significance depends on the T-statistic (therfore on the variance and the coefficient). But I don't understand why a component which explains more variance than another component gets insignificant and the other component (which explains less variance) gets significant. It must depend somehow on the variance of the variables that load on this component? Am I right? $\endgroup$– user36353Dec 18, 2013 at 10:50
-
3$\begingroup$ There's no reason why the twentieth principal component shouldn't predict a response perfectly & none of the others predict it at all. It's only your assumption that explaining more variance among the (suitably scaled) predictors in your sample should be anything to do with predicting the response. $\endgroup$– Scortchi - Reinstate Monica ♦Dec 18, 2013 at 12:16
-
3$\begingroup$ Imagine an example: your dependent var is dichotomous, two groups. And the scatterplot of these data in the space of some two variables X and Y shows that the groups are two separated oblong parallel clouds (like "="). You do PCA of X and Y variables and obtain PC1 (stronger one) parallel to the two clouds and PC2 (weaker one) perprendicular to it. Well, now, you see that PC1 is unable to predict (discriminate) the groups, but the weaker PC2 can do it well. $\endgroup$– ttnphnsDec 18, 2013 at 13:53
-
1$\begingroup$ Or imagine carrying out PCA on people's standardized weight & height. Which principal component would you expect to best predict high blood pressure? $\endgroup$– Scortchi - Reinstate Monica ♦Dec 18, 2013 at 14:47
Add a comment
|