Return to Answer

Let's first consider the simplest case of only one predictor (independent) variable $x$. For simplicity, let both $x$ and $y$ be centered, i.e. intercept is always zero. The difference between standard OLS regression and "orthogonal" TLS regression is clearly shown on this (adapted by me) figure from the most popular answer in the most popular thread on PCA:

Let's first consider the simplest case of only one predictor (independent) variable $x$. For simplicity, let both $x$ and $y$ be centered, i.e. intercept is always zero. The difference between standard OLS regression and "orthogonal" TLS regression is clearly shown on this (adapted by me) figure from the most popular answer in the most popular thread on PCA:

 v = pca([x y]);    //# x and y are centered column vectors
 beta = v(1,2)/v(1,1);

 v <- prcomp(cbind(x,y))$rotation
 beta <- v[1,2]/v[1,1]

 v = pca([x y]);    //# x and y are centered column vectors
 beta = v(2,1)/v(1,1);

 v <- prcomp(cbind(x,y))$rotation
 beta <- v[2,1]/v[1,1]

 v = pca([X y]);    //# X is a centered n-times-p matrix, y is n-times-1 column vector
 beta = v(1:end-1,end)/v(end,end);

 v <- prcomp(cbind(X,y))$rotation
 beta <- v[-ncol(v),ncol(v)] / v[ncol(v),ncol(v)]

 v = pca([X y]);    //# X is a centered n-times-p matrix, y is n-times-1 column vector
 beta = -v(1:end-1,end)/v(end,end);

 v <- prcomp(cbind(X,y))$rotation
 beta <- -v[-ncol(v),ncol(v)] / v[ncol(v),ncol(v)]