How is the first column of the matrix orthogonal to all the others

Question

$$ \mathbf{X}_{n\times(r+1)} = \begin{bmatrix} 1 & (x_{11}-\bar x_1) &\cdots & (x_{1r}-\bar x_r) \\ 1 &(x_{21}-\bar x_1) &\cdots & (x_{2r}-\bar x_r) \\ \vdots & \vdots &\ddots &\vdots\\ 1 &(x_{n1}-\bar x_1) &\cdots & (x_{nr}-\bar x_r) \\ \end{bmatrix} $$

The first column is supposed to be orthogonal to all the others, but I can't see why this is true.

Answer 1

The test of orthogonality is the dot product. The column of ones doesn't change any of the values in the other columns. So you are left with the summation of the distance of every point in any of the remaining columns to the mean of the column, which is zero by definition.

$$\begin{bmatrix}1&1\cdots&1\end{bmatrix} \begin{bmatrix}(X_{11}−\bar{X}_1)\\(X_{12}−\bar{X}_1)\\\vdots\\(X_{1n}−\bar{X}_1)\end{bmatrix}= \sum_{i=1}^{n}(X_{1i}−\bar{X}_1) = n\,(\bar{X}_{1} -\bar{X}_{1})=0 $$

Since, $$\sum_{i=1}^{n}(X_{1i})=n\bar{X}_{1}.$$

Same for other columns.

Answer 2

Two vectors $\mathbf{u}, \mathbf{v}$ are said to be orthogonal iff $$ \langle \mathbf{u}, \mathbf{v} \rangle = 0 $$ If you do this inner product $r$ times with $\mathbf{v}$ being column 1, and $\mathbf{u}$ being columns $2, ..., r+1 $ respectively, you just have to add the column elements. The orthogonality follows from the properties of the arithmetic mean. If you recall, its second property is that the sum of the deviations of the sample from the mean is zero.

Proof Let $\mathbf{x} = (x_1, \dots, x_n)'$ be the observed sample of the r.v. $X$ and let $\bar{x} = n^{-1} \mathbf{x}'\boldsymbol 1$ be the sample mean. Then $$ \sum_{i=1}^n (x_i - \bar{x}) = n \bar{x} - n \bar{x} = 0$$

In your case you just have samples $\mathbf{x}_1 = (x_{11}, \dots, x_{n1})', \dots, \mathbf{x}_r = (x_{1r}, \dots, x_{nr})'$ and the corresponding means are $\bar{x}_1, \dots, \bar{x}_r $. You can apply the same property taking care of adding the sample subscript:

$$ \sum_{i=1}^n (x_{ij} - \bar{x}_j) = n \bar{x}_j - n \bar{x}_j = 0 \qquad j=1,\dots, r.$$

I'll change the notation of your question to make it clearer. Hope this will be useful!