I have read in The Elements of Statistical Learning book and particularly in the Partial Least Squares (PLS) section:

Orthogonalize each $x_j^{(m−1)}$ with respect to $z_m$.

I would like to know what "orthogonalize" means in this statement or general.

1- what orthogonal means?
2- how to orthogonalise?

  • 3
    $\begingroup$ Could you expand your question by including the book's definitions for $x_j$ and $z_m$? $\endgroup$ – Alex R. Aug 20 '16 at 17:44
  • 2
    $\begingroup$ "Orthogonalize" most commonly refers to something along the lines of the Gram-Scmidt process (computationally, some variant of QR decomposition would typically be used). $\endgroup$ – GeoMatt22 Aug 20 '16 at 17:58
  • $\begingroup$ Two vectors are orthogonal if their dot product is zero. $\endgroup$ – bdeonovic Aug 21 '16 at 12:22
  • 1
    $\begingroup$ Can you edit your question to specify whether you want to know (a) how to orthogonalise (b) what orthogonal means (c) why in this application the author(s) want to orthogonalise? $\endgroup$ – mdewey Aug 21 '16 at 12:25

I believe the quote refers to this algorithm, where the relevant line reads:

$x_j^m=x_j^{m-1}-\frac{\langle z_m,x_j^{m-1}\rangle}{\langle z_m,z_m\rangle}z_m$

Here the authors are using the angle-brackets to denote an inner product, which is essentially the standard vector dot product from Physics 101.

The second term is the orthogonal projection of $x_j^{m-1}$ onto $z_m$. By subtracting this from $x_j^{m-1}$, the result $x_j^m$ is made orthogonal to $z_m$.

  • 2
    $\begingroup$ +1 Often in statistics this operation would be called "taking the residual of $x^{(m-1)}$ in the regression against $z$." $\endgroup$ – whuber Aug 22 '16 at 12:51
  • $\begingroup$ @ whuber you mean the orthogonal operation $\endgroup$ – Peter Smith Aug 24 '16 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.