# What does “orthogonalize” mean?

I have read in The Elements of Statistical Learning book and particularly in the Partial Least Squares (PLS) section:

Orthogonalize each $x_j^{(m−1)}$ with respect to $z_m$.

I would like to know what "orthogonalize" means in this statement or general.

1- what orthogonal means?
2- how to orthogonalise?

• Could you expand your question by including the book's definitions for $x_j$ and $z_m$? – Alex R. Aug 20 '16 at 17:44
• "Orthogonalize" most commonly refers to something along the lines of the Gram-Scmidt process (computationally, some variant of QR decomposition would typically be used). – GeoMatt22 Aug 20 '16 at 17:58
• Two vectors are orthogonal if their dot product is zero. – bdeonovic Aug 21 '16 at 12:22
• Can you edit your question to specify whether you want to know (a) how to orthogonalise (b) what orthogonal means (c) why in this application the author(s) want to orthogonalise? – mdewey Aug 21 '16 at 12:25

## 1 Answer

I believe the quote refers to this algorithm, where the relevant line reads:

$x_j^m=x_j^{m-1}-\frac{\langle z_m,x_j^{m-1}\rangle}{\langle z_m,z_m\rangle}z_m$

Here the authors are using the angle-brackets to denote an inner product, which is essentially the standard vector dot product from Physics 101.

The second term is the orthogonal projection of $x_j^{m-1}$ onto $z_m$. By subtracting this from $x_j^{m-1}$, the result $x_j^m$ is made orthogonal to $z_m$.

• +1 Often in statistics this operation would be called "taking the residual of $x^{(m-1)}$ in the regression against $z$." – whuber Aug 22 '16 at 12:51
• @ whuber you mean the orthogonal operation – Peter Smith Aug 24 '16 at 17:09