I have a simple matrix:

     [,1] [,2] [,3]
[1,]    1    2    3
[2,]    4    5    6
[3,]    7    8    9
[4,]   10   11   12

I have to calculate linear regression and orthogonal regression using lm() and prcomp() respectively. (for orthogonal see: here)

Assume that the first column is the the X and M the matrix I wrote before.


mod1 <- lm(M[,1] ~ M[,2] + M[,3] + 0)

Its output is (coefficient):

Coefficients: M[, 2]  M[, 3]  
     2      -1

Ok, I have these coefficients.

Now for


mod2 <- prcomp(~ M[,1] + M[,2] + M[,3])

Its output is:

             PC1        PC2        PC3
M[, 1] 0.5773503  0.0000000  0.8164966
M[, 2] 0.5773503 -0.7071068 -0.4082483
M[, 3] 0.5773503  0.7071068 -0.4082483

The question is: out to interpret prcomp() result instead of lm() result ? Using lm() the coefficients are using to predict the X values.

What about prcomp() ?

Thank you!

  • $\begingroup$ I also think you need to confirm you have quoted the correct coefficients for the lm(). I get -1, 1, NA for the data/model you show, which is correct given the linear dependencies in the example data. Don't forget that the intercept is also a coefficient, so your model actually has three coefficients. $\endgroup$ Jul 27, 2011 at 10:56
  • $\begingroup$ Gavin, My mistake, i added + 0 to have 0 intercept $\endgroup$
    – Dail
    Jul 27, 2011 at 11:00
  • 2
    $\begingroup$ prcomp is not doing orthogonal regression, at most can be used as an element of its implementation. Not to mention it is probably not worth it. $\endgroup$
    – user88
    Jul 27, 2011 at 11:55
  • $\begingroup$ @ mbq did you see the list i posted? It talk about orthogonal regression using prcomp()....btw what is the function to calculate the orthogonal regression in R? $\endgroup$
    – Dail
    Jul 27, 2011 at 12:00
  • $\begingroup$ You might want to start by reading the highly-rated thread on PCA at stats.stackexchange.com/questions/2691/…. $\endgroup$
    – whuber
    Jul 27, 2011 at 13:43

1 Answer 1


Although you have used $M_2$ and $M_3$ in your example, you effectively have an intercept because $M_2+1=M_3$, and so $$M_{1}=M_{2}\beta_2+M_{3}\beta_3=M_{2}\beta_2+(M_{2}+1)\beta_3=M_{2}(\beta_2+\beta_3)+\beta_3$$ (This is confirmed in @Gavin's comment, as $\beta_3=-1$ and $\beta_2+\beta_3=1$.). So your coefficient for $M_3$ is the intercept for the model with only $M_2$, and your coefficient for $M_2$ is the negative intercept for the model with only $M_3$.

In terms of a general comparison, you are comparing two straight lines (or points on straight lines). For a lm() model, we have


This line will pass through the point $(\beta_0,0,0,\dots,0)$ and have slopes in each direction of $(1,\beta_1,\beta_2,\dots,\beta_p)$. We can equivalently state this as passing through the point $(\beta_0+\overline{x}_1\beta_{1}+\dots+\overline{x}_p\beta_{p},\overline{x}_1,\dots,\overline{x}_p)$

Now in order to put this into the principal components analysis framework. Note that here we are only looking at the first PC. You are given a vector in $p+1$ dimensional space, $(\alpha_{Y},\alpha_{1},\dots,\alpha_{p})$, which is the principal component. This component describes a straight line, which passes through the centroid of the data $(\overline{y},\overline{x}_1,\dots,\overline{x}_p)$ and has slopes in each direction of $(\alpha_Y,\alpha_1,\dots,\alpha_p)$. Because slopes only need be defined in proportion (just alters the size of the line), we can restate this as $(1,\frac{\alpha_1}{\alpha_Y},\dots,\frac{\alpha_p}{\alpha_Y})$ (for $\alpha_Y\neq 0$ of course). This means $\beta_j$ is the equivalent quantity to $\frac{\alpha_{j}}{\alpha_Y}$. This is how you can compare the results

  • $\begingroup$ I am not sure I understand how OLS/lm() produces a "line". It fits the equation $$y_{i}=\beta_0+{x}_{i1}\beta_{1}+\dots+x_{ip}\beta_{p},$$ as you wrote, but how is that a line? Isn't it a $p$-dimensional hyperplane? $\endgroup$
    – amoeba
    Feb 3, 2015 at 15:13
  • $\begingroup$ In continuation of my previous comment: I posted an answer in the (arguably duplicate) thread that discusses PCA and TLS. See here: How to perform orthogonal regression (total least squares) via PCA? I continue to believe that your answer, as it is formulated now, is wrong. In fact, in multiple TLS regression one needs to look at the last and not at the first PCA eigenvector, in order to obtain the beta coefficients. $\endgroup$
    – amoeba
    Feb 6, 2015 at 22:32

Not the answer you're looking for? Browse other questions tagged or ask your own question.