0
$\begingroup$

I would be grateful if someone gives me some advice regarding the difference between the results that we got from the regression with original variables and regression with PCA components.

iris.train <- iris.train[,1:4]
A <- lm(Sepal.Length~., data=iris.train)
summary(A)

iris.train. <- iris.train[,-1]
pc <- prcomp(iris.train., retx=T)
summary(pc)
$\endgroup$
4
  • 3
    $\begingroup$ Could you be more specific? After all, if you use all the PCA components you are using all the regressors and you are fitting exactly the same model, but merely expressing it differently. What, then, might you be concerned about? Interpretability? Computational efficiency or stability? What happens when using a proper subset of the components? Something else? $\endgroup$
    – whuber
    Jun 15, 2017 at 21:57
  • $\begingroup$ @whuber, thanks so much for your comment. First I would like to know how we can interpret the results ofter applying regression on components instead of original variables. Second how we can figure out which model (the one which original variables vs the ones with components) is more accurate? $\endgroup$
    – zara
    Jun 15, 2017 at 23:49
  • 1
    $\begingroup$ As I mentioned, if you use all the components, the models will have identical fits. You can confirm that in your code. $\endgroup$
    – whuber
    Jun 16, 2017 at 1:03
  • $\begingroup$ @zara You would have to expand ouf the PCs times each coefficient and add them up to do what whuber is describing. But it does like $a*PC1 + b*PC2 + c*PC3 = a*(w1x1 + w2x2 + w3x3) + b*(w1x1 + w2x2 + w3x3) + c*(w1x1 + w2x2 + w3x3)$. Then do the algebra to put it all in terms of $x1$, $x2$, and $x3$ and compare to the regression on the original $x1$, $x2$, and $x3$. (Here, $w1$, $w2$, and $w3$ are the weights on each original variable that you get from the eigenvectors. There should be nine weights, not three, but you get the idea.) $\endgroup$
    – Dave
    Sep 13, 2019 at 10:45

1 Answer 1

0
$\begingroup$

Partially answered in comments:

First I would like to know how we can interpret the results after applying regression on components instead of original variables. Second how we can figure out which model (the one which original variables vs the ones with components) is more accurate?

– zara

to which the answer is

As I mentioned, if you use all the components, the models will have identical fits. You can confirm that in your code.

– whuber

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.