7
$\begingroup$

I have read in The Elements of Statistical Learning book and particularly in the Partial Least Squares (PLS) section:

Orthogonalize each $x_j^{(m−1)}$ with respect to $z_m$.

I would like to know what "orthogonalize" means in this statement or general.

1- what orthogonal means?
2- how to orthogonalise?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Could you expand your question by including the book's definitions for $x_j$ and $z_m$? $\endgroup$
    – Alex R.
    Commented Aug 20, 2016 at 17:44
  • 2
    $\begingroup$ "Orthogonalize" most commonly refers to something along the lines of the Gram-Scmidt process (computationally, some variant of QR decomposition would typically be used). $\endgroup$
    – GeoMatt22
    Commented Aug 20, 2016 at 17:58
  • $\begingroup$ Two vectors are orthogonal if their dot product is zero. $\endgroup$
    – bdeonovic
    Commented Aug 21, 2016 at 12:22
  • 1
    $\begingroup$ Can you edit your question to specify whether you want to know (a) how to orthogonalise (b) what orthogonal means (c) why in this application the author(s) want to orthogonalise? $\endgroup$
    – mdewey
    Commented Aug 21, 2016 at 12:25

1 Answer 1

Reset to default
6
$\begingroup$

I believe the quote refers to this algorithm, where the relevant line reads:

$x_j^m=x_j^{m-1}-\frac{\langle z_m,x_j^{m-1}\rangle}{\langle z_m,z_m\rangle}z_m$

Here the authors are using the angle-brackets to denote an inner product, which is essentially the standard vector dot product from Physics 101.

The second term is the orthogonal projection of $x_j^{m-1}$ onto $z_m$. By subtracting this from $x_j^{m-1}$, the result $x_j^m$ is made orthogonal to $z_m$.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ +1 Often in statistics this operation would be called "taking the residual of $x^{(m-1)}$ in the regression against $z$." $\endgroup$
    – whuber ♦
    Commented Aug 22, 2016 at 12:51
  • $\begingroup$ @ whuber you mean the orthogonal operation $\endgroup$
    – Peter Smith
    Commented Aug 24, 2016 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.