How do we get that $\hat\beta_1\,\sigma_x = \widehat{\beta_1\sigma_x} = \frac{1}{n}\sum_{i=1}^n \xi_i y_i = \sum_{i=1}^n \frac{\xi_i}{n} y_i$?

Question

I am trying to understand this answer by @whuber to this question of how to get the standard errors of the regression estimators. @whuber says the following:

Variance of the slope estimate

The Ordinary Least Squares estimate of the slope in these new units is simply the average product of the $\xi_i$ and $y_i,$

$$\hat\beta_1\,\sigma_x = \widehat{\beta_1\sigma_x} = \frac{1}{n}\sum_{i=1}^n \xi_i y_i = \sum_{i=1}^n \frac{\xi_i}{n} y_i.\tag{2}$$

I can see that we can write the standardised value as

$$\dfrac{x_i - \overline{x}}{\sigma_x} = \zeta_i.$$

So the slope in the new units, $\beta_1 \zeta_i$, is the old slope, $\beta_1$, scaled by the standard deviation of all the $x$, $\sigma_x$. With that said, I still don't see where this idea that $\hat{\beta_1 \sigma_x} = \dfrac{1}{n} \sum\limits_{i = 1}^n \zeta_i y_i$ is coming from.

@whuber then says that $\hat{\beta_1 \sigma_x} = \dfrac{1}{n} \sum\limits_{i = 1}^n \zeta_i y_i$ comes from the normal equation of least squares. Following the link in @whuber's answer here to this question, I fully write out a small example of $(X^T X)\beta = X^T Y$, where $n = 2$ and $p = 2$ (see here), to get

$$\begin{bmatrix} 2 \beta_0 + (x_{11} + x_{21})\beta_1 + (x_{12} + x_{22}) \beta_2 \\ (x_{11} + x_{21})\beta_0 + (x_{11}^2 + x_{21}^2)\beta_1 + (x_{11}x_{12} + x_{21}x_{22})\beta_2 \\ (x_{12} + x_{22})\beta_0 + (x_{11}x_{12} + x_{21}x_{22})\beta_1 + (x_{12}^2 + x_{22}^2)\beta_2 \end{bmatrix} = \begin{bmatrix} (\beta_0 + \beta_1 x_{11} + \beta_2 x_{12}) + (\beta_0 + \beta_1 x_{21} + \beta_2 x_{22}) \\ x_{11} (\beta_0 + \beta_1 x_{11} + \beta_2 x_{12}) + x_{21}(\beta_0 + \beta_1 x_{21} + \beta_2 x_{22}) \\ x_{12}(\beta_0 + \beta_1 x_{11} + \beta_2 x_{12}) + x_{22}(\beta_0 + \beta_1 x_{21} + \beta_2 x_{22}) \end{bmatrix}$$

This seems to be correct. But, even so, I don't see how this clarifies $\hat{\beta_1 \sigma_x} = \dfrac{1}{n} \sum\limits_{i = 1}^n \zeta_i y_i$.

So how do we get that $\hat\beta_1\,\sigma_x = \widehat{\beta_1\sigma_x} = \frac{1}{n}\sum_{i=1}^n \xi_i y_i = \sum_{i=1}^n \frac{\xi_i}{n} y_i$?

Answer 1

It stems from the expression for the estimate of the slope-coefficient of the simple linear regression

$$\hat{\beta}_1 = \dfrac{\sum\limits_{i = 1}^{n}(x_i - \overline{x})(y_i - \overline{y})}{\sum\limits_{i = 1}^{n}(x_i - \overline{x})^2}$$

Or in the new coordinates using $\xi_i$ instead of $x_i$

$$\widehat{\beta_1 \sigma_x} = \dfrac{\sum\limits_{i = 1}^{n}(\xi_i - \overline{\xi})(y_i - \overline{y})}{\sum\limits_{i = 1}^{n}(\xi_i - \overline{\xi})^2}$$

And these $\xi_i$ where cleverly defined such that

$$\begin{array}{rcl} \sum\limits_{i = 1}^{n}(\xi_i - \overline{\xi})^2 &=& n \\ \bar \xi &=& 0 \end{array}$$

If you fill those into the expression for $ \widehat{\beta_1 \sigma_x}$ then you get to the simpler expression for the estimate

$$\begin{array}{} \widehat{\beta_1 \sigma_x} &=& \dfrac{\sum\limits_{i = 1}^{n}(\xi_i - \overline{\xi})(y_i - \overline{y})}{\sum\limits_{i = 1}^{n}(\xi_i - \overline{\xi})^2} \\ && \text{filling in / substituting} \\ &=& \frac{\sum\limits_{i = 1}^{n}\xi_i(y_i - \overline{y})}{n} \\ && \text{splitting the product with a sum into a sum of two products} \\ &=& \frac{{\sum\limits_{i = 1}^{n}\xi_i y_i } + \sum\limits_{i = 1}^{n}- \xi_i \overline{y}}{n} \\ && \text{move term $\bar{y}$ outside the sum (it's not dependent on $i$)} \\ &=& \frac{{\sum\limits_{i = 1}^{n}\xi_i y_i } - \overline{y} \sum\limits_{i = 1}^{n} \xi_i}{n} \\&& \text{the sum of $\xi_i$ is zero} \\ &=& \frac{\sum\limits_{i = 1}^{n}\xi_i y_i }{n} \\&& \text{move the division by $n$} \\ &=& \sum\limits_{i = 1}^{n}\frac{\xi_i}{n} y_i \end{array}$$