# Show that $\text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\sum_{i=1}^{n}(x_{i1}-\overline{x}_{i})^{2}(1-r^{2})}$

For $$i = 1,2,\ldots, n$$, consider \begin{align*} Y_{i} = \beta_{0} + \beta_{1}(x_{i1} - \overline{x}_{1}) + \beta_{2}(x_{i2} - \overline{x}_{2}) + \epsilon_{i} \end{align*} where $$\overline{x}_{j} = \sum_{i=1}^{n}x_{ij}/n$$, $$\textbf{E}[\epsilon] = 0$$ and $$\textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n}$$. If $$\hat{\beta}_{1}$$ is the least squares estimate of $$\beta_{1}$$, show that \begin{align*} \text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\sum_{i=1}^{n}(x_{i1}-\overline{x}_{i})^{2}(1-r^{2})} \end{align*}

where $$r$$ is the correlation coefficient of the pairs $$(x_{i1},x_{i2})$$.

MY ATTEMPT

So far I have obtained the following relations \begin{align*} \textbf{X} = \begin{bmatrix} 1 & x_{11} & x_{12} \\ 1 & x_{21} & x_{22} \\ \vdots & \vdots & \vdots \\ 1 & x_{n1} & x_{n2}\\ \end{bmatrix}\quad\wedge\quad \textbf{Z} = \begin{bmatrix} 0 & \overline{x}_{1} & \overline{x}_{2} \\ 0 & \overline{x}_{1} & \overline{x}_{2} \\ \vdots & \vdots & \vdots \\ 0 & \overline{x}_{1} & \overline{x}_{2}\\ \end{bmatrix} \end{align*}

Consequently, the proposed model can be rewritten as \begin{align*} \textbf{Y} = (\textbf{X}-\textbf{Z})\beta + \epsilon = \textbf{W}\beta + \epsilon \end{align*}

But I do not know how to proceed from here. Can someone help me out? Thanks in advance!

EDIT

After some thought, I came up with the following solution \begin{align*} \textbf{W} & = \begin{bmatrix} 1 & x_{11} - \overline{x}_{1} & x_{12} - \overline{x}_{2} \\ 1 & x_{21} - \overline{x}_{1} & x_{22} - \overline{x}_{2} \\ \vdots & \vdots & \vdots \\ 1 & x_{n1} - \overline{x}_{1} & x_{n2} - \overline{x}_{2}\\ \end{bmatrix} \Longrightarrow\\\\ \textbf{W}^{\prime}\textbf{W} & = \begin{bmatrix} n & 0 & 0 \\ 0 & \displaystyle\sum_{i=1}^{n}(x_{i1}-\overline{x}_{1})^{2} & \displaystyle\sum_{i=1}^{n}(x_{i1}-\overline{x}_{1})(x_{i2}-\overline{x}_{2}) \\ 0 & \displaystyle\sum_{i=1}^{n}(x_{i1}-\overline{x}_{1})(x_{i2}-\overline{x}_{2}) & \displaystyle\sum_{i=1}^{n}(x_{i2}-\overline{x}_{1})^{2} \\ \end{bmatrix}\\\\ (\textbf{W}^{\prime}\textbf{W})^{-1} & = \frac{1}{\det(\textbf{W}^{\prime}\textbf{W})} \begin{bmatrix} n(s_{xx}s_{yy}-s^{2}_{xy}) & 0 & 0 \\ 0 & ns_{xx}s_{yy} & -ns^{2}_{xy} \\ 0 & -ns^{2}_{xy} & ns_{xx}s_{yy} \\ \end{bmatrix} \end{align*}

where $$\det(\textbf{W}^{\prime}\textbf{W}) = n(s_{xx}s_{yy} - s^{2}_{xy})$$. Once \begin{align*} \textbf{Var}[\textbf{Y}] & = \textbf{Var}[\textbf{Y} - \textbf{W}\beta] = \textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n} \Longrightarrow\\\\ \textbf{Var}[\hat{\beta}] & = \textbf{Var}[(\textbf{W}^{\prime}\textbf{W})^{-1}\textbf{W}^{\prime}\textbf{Y}] = (\textbf{W}^{\prime}\textbf{W})^{-1}\textbf{W}^{\prime}\textbf{Var}[\textbf{Y}]\textbf{W}(\textbf{W}^{\prime}\textbf{W})^{-1}\\\\ & = \sigma^{2}(\textbf{W}^{\prime}\textbf{W})^{-1}\textbf{W}^{\prime}\textbf{W}(\textbf{W}^{\prime}\textbf{W})^{-1} = \sigma^{2}(\textbf{W}^{\prime}\textbf{W})^{-1} \end{align*} However such reasoning leads to the expression \begin{align*} \text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\displaystyle 1 - \frac{s^{2}_{xy}}{s_{xx}s_{yy}}} = \frac{\sigma^{2}}{1 - r^{2}} \end{align*}

Can someone tell me where is the mistake? Thanks in advance!

EDIT

I think I found the mistake: I was miscalculating the inverse of $$\textbf{W}^{\prime}\textbf{W}$$. Indeed, we have

\begin{align*} (\textbf{W}^{\prime}\textbf{W})^{-1} & = \frac{1}{\det(\textbf{W}^{\prime}\textbf{W})} \begin{bmatrix} s_{xx}s_{yy}-s^{2}_{xy} & 0 & 0 \\ 0 & ns_{yy} & -ns_{xy} \\ 0 & -ns_{xy} & ns_{xx} \\ \end{bmatrix} \end{align*}

\begin{align*} \therefore \text{Var}(\hat{\beta}_{1}) = \frac{n\sigma^{2}s_{yy}}{n(s_{xx}s_{yy} - s^{2}_{xy})} = \frac{\sigma^{2}}{\displaystyle s_{xx} - \frac{s^{2}_{xy}}{s_{yy}}} = \frac{\sigma^{2}}{\displaystyle s_{xx}\left(1 - \frac{s^{2}_{xy}}{s_{xx}s_{yy}}\right)} = \frac{\sigma^{2}}{s_{xx}(1-r^{2})} \end{align*}

I think it is fine now :)

• apart from small typos that doesn't affect the final result, yes, it seems you've provided a nearly complete solution :) – gunes Apr 2 at 6:59