2
$\begingroup$

I have a set of five variables that are highly correlated (namely SAPS3, SIDA, ALBUM, Lactate), and I must see how they are associated with other variables (I will use a variable named lIL6 as an example).

I have decided to use principal component analysis to deal with collinearity and try to understand how these variables behave.
I decided to use three PC (all with eigenvalues above one). Together, they explain at least 70% of the variance (which is fine for me).
I used the psych package (dbp is the name of the database):

ppp.r<-principal(dbp,nfactors=3,rotate="varimax",scores=T)

This returns the following:

Call: principal(r = dbp, nfactors = 3, rotate = "varimax", scores = T)
Standardized loadings (pattern matrix) based upon correlation matrix

          RC1   RC2   RC3   h2    u2
SAPS3   -0.09  0.09  0.98 0.97 0.026
Albumin  0.53 -0.70 -0.12 0.78 0.220
SIDa     0.88  0.04  0.07 0.78 0.217
SIG      0.16  0.92  0.05 0.88 0.119
Lactate -0.65  0.04  0.19 0.46 0.543

                       RC1  RC2  RC3
SS loadings           1.51 1.35 1.02
Proportion Var        0.30 0.27 0.20
Cumulative Var        0.30 0.57 0.78
Proportion Explained  0.39 0.35 0.26
Cumulative Proportion 0.39 0.74 1.00

I have the following doubts:

1- What is the optimal cutoff to say that a variable has a negligible load on the PC score? I have read 0.50 in some places, while others said 0.60. Can I say, for instance, that the load of SAPS3 on PC1 is negligible? What about the load of Albumin on PC1 (it is 0.53)?

2- Since I want to see how these variables are related to another variable (variable lIL6), I therefore performed a simple linear regression between PC and lIL6. I am particularly interested in PC2 because SIG and Albumin have a high load on it, so I ran this:

lm(formula = lIL6 ~ ppp.r$scores[, 2], data = db)  

Residuals:  
    Min      1Q  Median      3Q     Max   
-1.6547 -0.5058 -0.1231  0.4078  2.3494   

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)     
(Intercept)        1.61663    0.08818  18.332  < 2e-16 ***  
ppp.r$scores[, 2]  0.32542    0.08870   3.669 0.000424 ***  

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1   

 Residual standard error: 0.8225 on 85 degrees of freedom  
 Multiple R-squared: 0.1367,    Adjusted R-squared: 0.1266   
 F-statistic: 13.46 on 1 and 85 DF,  p-value: 0.0004239   

So it is significant (I understand that it is not a good model, R-squared is low, etc) Can I state that, since SIG and Albumin have a high load in PC2 and since PC2 is associated with lIL6, SIG and Albumin are probably associated with lIL6 in a manner that is independent of SAPS3 values (considering that SAPS3 has a very low load on PC2)?

$\endgroup$
  • 1
    $\begingroup$ What problem(s) do you run into when you simply regress lILC6 against the original five variables? Since the third PC still accounts for 20% of all variance, with 100-78 = 22% left to explain, none of the remaining two PCs (not shown) can have more than 20%, whence each has at least 2% (and likely more): this implies the eigenvectors don't vary a whole lot--the ratio of the highest to the lowest is at at most $\sqrt{30\%/2\%} \lt 4$, indicating there should be no multicollinearity problem at all! $\endgroup$ – whuber Nov 18 '13 at 21:40
1
$\begingroup$

Answered partially in comments:

What problem(s) do you run into when you simply regress lILC6 against the original five variables? Since the third PC still accounts for 20% of all variance, with 100-78 = 22% left to explain, none of the remaining two PCs (not shown) can have more than 20%, whence each has at least 2% (and likely more): this implies the eigenvectors don't vary a whole lot--the ratio of the highest to the lowest is at at most $\sqrt{30\%/2\%} \lt 4$, indicating there should be no multicollinearity problem at all! – whuber

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.