I just want to ask a question about notation in this exercise.
In Equation $X^*_{.,i} =M_{X.,-i} * X_i$ ;
$X^*_{.,i}$ means ith column of original matrix $M_{X.,-i}$ means orthogonal projection matrix of the column space $X^*_{.,-i}$(Every column except ith) But what does $X_i$ mean? Is it the same as X*.,i (so $X^*_{.,i} = X_i$)?
Following the notation that's used in the passage:
$$ X = \begin{bmatrix} X_{1} & X_{2} & \ldots & X_{k}\end{bmatrix} $$ That is $X_{i}$ is a column vector for the $i$th column of matrix $X$.
$ X_{.,-i}$ is the matrix $X$ with the $i$th column removed. It is an $n$ by $k-1$ matrix. For example: $$ X_{.,-2} = \begin{bmatrix}X_{1} & X_3 & X_4 & \ldots & X_{k}\end{bmatrix} $$
The $n$ by $n$ matrix $M_{X_{.,-i}}$ is defined as: $$M_{X_{.,-i}} = I - X_{.,-i}\left( X_{.,-i}'X_{.,-i}\right)^{-1}X_{.,-i}'$$ The $n$ by $1$ vector $X^*_{.,i}$ is defined as: $$X^*_{.,i} = M_{X_{.,-i}} X_i$$
The vectors $X_i$ and $X^*_{.,i}$ are not the same! You'll find that vector $X^*_{.,i}$ is vector $X_i$ minus the projection of $X_i$ onto the linear space spanned by all the column vectors $X_1$, $X_2$, etc... besides $X_i$.