# prcomp() vs lm() results in R

I have a simple matrix:

     [,1] [,2] [,3]
[1,]    1    2    3
[2,]    4    5    6
[3,]    7    8    9
[4,]   10   11   12


I have to calculate linear regression and orthogonal regression using lm() and prcomp() respectively. (for orthogonal see: here)

Assume that the first column is the the X and M the matrix I wrote before.

LINEAR REG.

mod1 <- lm(M[,1] ~ M[,2] + M[,3] + 0)


Its output is (coefficient):

Coefficients: M[, 2]  M[, 3]
2      -1


Ok, I have these coefficients.

Now for

ORTHOGONAL REG.

mod2 <- prcomp(~ M[,1] + M[,2] + M[,3])


Its output is:

             PC1        PC2        PC3
M[, 1] 0.5773503  0.0000000  0.8164966
M[, 2] 0.5773503 -0.7071068 -0.4082483
M[, 3] 0.5773503  0.7071068 -0.4082483


The question is: out to interpret prcomp() result instead of lm() result ? Using lm() the coefficients are using to predict the X values.

Thank you!

-
I also think you need to confirm you have quoted the correct coefficients for the lm(). I get -1, 1, NA for the data/model you show, which is correct given the linear dependencies in the example data. Don't forget that the intercept is also a coefficient, so your model actually has three coefficients. – Gavin Simpson Jul 27 '11 at 10:56
Gavin, My mistake, i added + 0 to have 0 intercept – Dail Jul 27 '11 at 11:00
prcomp is not doing orthogonal regression, at most can be used as an element of its implementation. Not to mention it is probably not worth it. – mbq Jul 27 '11 at 11:55
@ mbq did you see the list i posted? It talk about orthogonal regression using prcomp()....btw what is the function to calculate the orthogonal regression in R? – Dail Jul 27 '11 at 12:00
You might want to start by reading the highly-rated thread on PCA at stats.stackexchange.com/questions/2691/…. – whuber Jul 27 '11 at 13:43

Although you have used $M_2$ and $M_3$ in your example, you effectively have an intercept because $M_2+1=M_3$, and so $$M_{1}=M_{2}\beta_2+M_{3}\beta_3=M_{2}\beta_2+(M_{2}+1)\beta_3=M_{2}(\beta_2+\beta_3)+\beta_3$$ (This is confirmed in @Gavin's comment, as $\beta_3=-1$ and $\beta_2+\beta_3=1$.). So your coefficient for $M_3$ is the intercept for the model with only $M_2$, and your coefficient for $M_2$ is the negative intercept for the model with only $M_3$.
$$y_{i}=\beta_0+{x}_{i1}\beta_{1}+\dots+x_{ip}\beta_{p}$$
This line will pass through the point $(\beta_0,0,0,\dots,0)$ and have slopes in each direction of $(1,\beta_1,\beta_2,\dots,\beta_p)$. We can equivalently state this as passing through the point $(\beta_0+\overline{x}_1\beta_{1}+\dots+\overline{x}_p\beta_{p},\overline{x}_1,\dots,\overline{x}_p)$
Now in order to put this into the principal components analysis framework. Note that here we are only looking at the first PC. You are given a vector in $p+1$ dimensional space, $(\alpha_{Y},\alpha_{1},\dots,\alpha_{p})$, which is the principal component. This component describes a straight line, which passes through the centroid of the data $(\overline{y},\overline{x}_1,\dots,\overline{x}_p)$ and has slopes in each direction of $(\alpha_Y,\alpha_1,\dots,\alpha_p)$. Because slopes only need be defined in proportion (just alters the size of the line), we can restate this as $(1,\frac{\alpha_1}{\alpha_Y},\dots,\frac{\alpha_p}{\alpha_Y})$ (for $\alpha_Y\neq 0$ of course). This means $\beta_j$ is the equivalent quantity to $\frac{\alpha_{j}}{\alpha_Y}$. This is how you can compare the results