How to derive the kth coefficient standard error?

Question

Given a multiple regression with the usual assumptions satisfied, with $X \in R^{n \times p}$

$$ y = X \beta + e $$

I know that the estimated variance is given by $\sigma^2 (X^TX)^{-1}$. But what I want to know is the estimated variance for the $k^{th}$ coefficient out of this covariance matrix.

I've seen from the resource A (cited below) that this value is the $k^{th}$ diagonal term of the $\sigma^2 (X^TX)^{-1}$. But more interestingly (AND THIS IS WHAT I WANT TO PROVE), this value is:

$$ \dfrac{\sigma^2}{(1-R^2_k) \sum_{i=1}^n (x_{ik} - \bar{x_k})^2 } $$

Here, $x_k$ is the $k^{th}$ column of $X$. $R^2_k$ is the $R^2$ from regressing $x_k$ on $X_{(k)}$(=$X$ after taking $k^{th}$ column out).

Here is my (failed) attempt at this derivation:

Using resource B (cited below), the $k^{th}$ diagonal value of $\sigma^2 (X^TX)^{-1}$ is:

$$\sigma^2 [x_k^Tx_k - x_k^T X_{(k)} (X_{(k)}^TX_{(k)})^{-1}X_{(k)}^T x_k ] ^ {-1}$$

Since $X_{(k)} (X_{(k)}^TX_{(k)})^{-1}X_{(k)}^T x_k$ can be seen as projection of $x_k$ onto column space of $X_{(k)}$, we can say:

$$ X_{(k)} (X_{(k)}^TX_{(k)})^{-1}X_{(k)}^T x_k = \hat{x_k} $$, which is the regression prediction after regressing $x_k$ on $X_{(k)}$.

So our diagonal value simplifies to:

$$ \sigma^2 [x_k^Tx_k - x_k^T X_{(k)} (X_{(k)}^TX_{(k)})^{-1}X_{(k)}^T x_k ] ^ {-1}= \sigma^2 [ x_k^Tx_k - x_k^T \hat{x_k} ] ^ {-1}$$

Now, since (I will omit subscript $i$ so $x_k = x_{ik}$)

$$ R^2_k = 1 - \dfrac{ \sum (x_k - \hat{x_k} )^2 }{ \sum (x_k - \bar{x_k} )^2 } $$

$$ \dfrac{\sigma^2}{(1-R^2_k) \sum (x_{k} - \bar{x_k})^2 } $$

will simplify to:

$$ \dfrac{\sigma^2}{ \sum (x_{k} - \hat{x_k})^2 } $$

So in summary, we want to show that:

$$ \sigma^2 [ x_k^Tx_k - x_k^T \hat{x_k} ] ^ {-1} = \dfrac{\sigma^2}{ \sum (x_{k} - \hat{x_k})^2 }$$

But then,

$$ \sigma^2 [ x_k^Tx_k - x_k^T \hat{x_k} ] ^ {-1} = \dfrac{\sigma^2}{\sum (x_k)^2 - \sum x_k \hat{x_k} } $$

where as,

$$ \dfrac{\sigma^2}{ \sum (x_{k} - \hat{x_k})^2 } = \dfrac{\sigma^2}{\sum (x_k)^2 - 2\sum x_k \hat{x_k} + \sum \hat{x_k}^2 } $$

Could someone please help me find where I am making a mistake, and if this is not the right approach, help me derive it correctly?

Resource A: http://people.stern.nyu.edu/wgreene/MathStat/GreeneChapter4.pdf (page 40)

Resource B: https://math.stackexchange.com/questions/1562745/about-the-diagonal-entries-of-an-inverse-matrix

Answer 1

I will answer my own question:

It comes down to proving $$ \sum x_k \hat{x_k} = \sum \hat{x_k} \hat{x_k} $$

We can see this because $ x_k = \hat{x_k} + e_k $, where $e_k$ is the residual.

$$ \sum x_k \hat{x_k} = \sum (\hat{x_k} + e_k) \hat{x_k} $$

$$ = \sum \hat{x_k} \hat{x_k} + \sum e_k \hat{x_k}$$

But then $ \sum e_k \hat{x_k} = 0 $ since fitted values and residuals are orthogonal.