3
$\begingroup$

I'm trying to understand Principal Component Regression(PCR) using dummy data (no multicollinearity). I have managed to regress dependent variable on scores but how do I convert the coefficients back to coefficients of original independent data? Below is my example using R

# create dummy data for testing
a1<-1:10
a2<-rnorm(10,0,5)  # uncorrelated to a1, for testing purposes
a<-cbind(a1,a2)
y<-a1-2*a2   # dependent variable

pr<-princomp(a,cor=TRUE)
s<-pr$scores
m2<-lm(y~s)

# results
m2$coefficients 
(Intercept)     sComp.1     sComp.2 
 2.461620   -7.370569  -11.432588

How do I convert m2$coefficients back to coefficients in terms of original data?

From what I have learned so far, I only need to multiply EigenVectors with the above coefficients. But the coefficients have dimension of 3x1 and eigenvectors are 2x2!

$\endgroup$
5
  • 1
    $\begingroup$ Note that the Intercept term is independent of the PCs/original variables. It's the other coefficients that you need to associate with the eigenvectors. $\endgroup$
    – EdM
    Jun 20, 2015 at 19:28
  • $\begingroup$ @EdM Thank you for your reply, I multiplied the coefficient of scores with eigen vectors but the results look very different from 1 and -2. Which is what they should be given there is little multicollinearity. Please correct me if I'm wrong. $\endgroup$ Jun 20, 2015 at 19:57
  • 1
    $\begingroup$ Post the code you used to do the multiplication of coefficients by the eigenvectors and the results; show the eigenvectors too. For reproducing your results, it helps to set (and report) the seed for the random number generator before the rnorm() command. I have a hunch that this has to do with centering and normalizing (or not) the data before the PCA. $\endgroup$
    – EdM
    Jun 20, 2015 at 20:21
  • $\begingroup$ @user2696565, please do as EdM suggested in their last comment if you want this question to be resolved. $\endgroup$
    – amoeba
    Jul 8, 2015 at 21:50
  • $\begingroup$ @EdM Thank you for your help, I should have replied earlier that it is now resolved. The solution was your first comment - that the intercept term s independent. I had a bug in my code when I wrote my previous comment. $\endgroup$ Jul 9, 2015 at 21:26

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.