# Use of PCA analysis to selection of variables for a regression analysis

I have too many environmental variables to use in a multiple regression analysis. If I use all the variables to models are just to complex. The use of the PCA axis in the regression analysis was impossible to interpret (since there wasn't a clear correlation to environmental variables), so we chose to select a limited number of variables, namely the ones that had the higher explanation in PCA... A PCA was used for each set of environmental variables, namely the variables related with the structure of the stream, the evolving vegetation, the climatic, the physical-chemical characteristics of the water from summer period and from winter period, separately. The PCA was performed using the correlation matrix option, using the software PC-ORD, v. 4.21 (McCune & Mefford 1999). For each set of variables, only the variables with coordinates higher to 0.20, for the two first axis, of the CA, were selected to be used in Multiple regression analysis. I could not found literature that confirmes that that's ok to do this, but I think it is not wrong. Can anyone help me?

Best regards, Leena

-
welcome to the website. What is your analysis goal? I can read two options between the lines. (1) You want to select the variables that contribute most into explaining the outcome. (2) You want to have all variables in the model, as you know that all of them matter, but you are concerned about multicollinearity and large standard errors that result. For either of these tasks, yours is suboptimal procedure (and frankly the rest of the world uses R, rather than some 12-year old package). – StasK Feb 29 '12 at 16:29
The 2012 study reported at r4stats.com/articles/popularity showed 31% of data analysts using R. – rolando2 Mar 11 at 23:29

When it comes to PCA, relatively few approaches are objectively wrong. But I see two potential problems with your approach. First, suppose a variable has no high loading on any obtained component. That means it is basically independent from the other variables in the analysis. Does this mean it will not contribute to your regression? Not necessarily. Suppose it fills "its own niche" very well in accounting for some of the variance in your outcome. After all, that outcome was not included in the PCA, right?

The use of the PCA axis in the regression analysis was impossible to interpret (since there wasn't a clear correlation to environmental variables), so we chose to select a limited number of variables, namely the ones that had the higher explanation in PCA.

The last part of this seems contradictory to me. If you want to use individual variables as predictors rather than the poorly-performing components that summarize dimensions in the data, the most consistent thing to do is to disregard the PCA results when selecting predictors. You care about how a predictor relates to the outcome, not about ensuring that different predictors are correlated with each other.

Another note: it sounds as if you have conducted or will conduct multiple iterations of regression as you attempt to find good predictors. You'll want to read, on this site or elsewhere, discussions of ways to avoid the pitfalls that come with that. I'd recommend looking at posts on this site with the tag "variable selection."

-

As @Rolando2 said, the whole area of variable selection comes up here a lot.

But one method that gets to the substantive problem he has with PCA is partial least squares; if you are using SAS, there is PROC PLS.

-

The most recent article I have seen on this problem is

• Cumming, J.A. and Wooff, D.A. (2007)Dimension reduction via principal variables, Computational Statistics & Data Analysis, 52, 1:550-565.

A good earlier review was provided in chapter 6 of

• Jolliffe, IT(2002). Principal Component Analysis. Springer-Verlag, New York, 2nd edition.
-
Thank you for those references. Would you mind summarizing what they say? – whuber Mar 11 at 20:34