1
$\begingroup$

For $i = 1,2,\ldots, n$, consider \begin{align*} Y_{i} = \beta_{0} + \beta_{1}(x_{i1} - \overline{x}_{1}) + \beta_{2}(x_{i2} - \overline{x}_{2}) + \epsilon_{i} \end{align*} where $\overline{x}_{j} = \sum_{i=1}^{n}x_{ij}/n$, $\textbf{E}[\epsilon] = 0$ and $\textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n}$. If $\hat{\beta}_{1}$ is the least squares estimate of $\beta_{1}$, show that \begin{align*} \text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\sum_{i=1}^{n}(x_{i1}-\overline{x}_{i})^{2}(1-r^{2})} \end{align*}

where $r$ is the correlation coefficient of the pairs $(x_{i1},x_{i2})$.

MY ATTEMPT

So far I have obtained the following relations \begin{align*} \textbf{X} = \begin{bmatrix} 1 & x_{11} & x_{12} \\ 1 & x_{21} & x_{22} \\ \vdots & \vdots & \vdots \\ 1 & x_{n1} & x_{n2}\\ \end{bmatrix}\quad\wedge\quad \textbf{Z} = \begin{bmatrix} 0 & \overline{x}_{1} & \overline{x}_{2} \\ 0 & \overline{x}_{1} & \overline{x}_{2} \\ \vdots & \vdots & \vdots \\ 0 & \overline{x}_{1} & \overline{x}_{2}\\ \end{bmatrix} \end{align*}

Consequently, the proposed model can be rewritten as \begin{align*} \textbf{Y} = (\textbf{X}-\textbf{Z})\beta + \epsilon = \textbf{W}\beta + \epsilon \end{align*}

But I do not know how to proceed from here. Can someone help me out? Thanks in advance!

EDIT

After some thought, I came up with the following solution \begin{align*} \textbf{W} & = \begin{bmatrix} 1 & x_{11} - \overline{x}_{1} & x_{12} - \overline{x}_{2} \\ 1 & x_{21} - \overline{x}_{1} & x_{22} - \overline{x}_{2} \\ \vdots & \vdots & \vdots \\ 1 & x_{n1} - \overline{x}_{1} & x_{n2} - \overline{x}_{2}\\ \end{bmatrix} \Longrightarrow\\\\ \textbf{W}^{\prime}\textbf{W} & = \begin{bmatrix} n & 0 & 0 \\ 0 & \displaystyle\sum_{i=1}^{n}(x_{i1}-\overline{x}_{1})^{2} & \displaystyle\sum_{i=1}^{n}(x_{i1}-\overline{x}_{1})(x_{i2}-\overline{x}_{2}) \\ 0 & \displaystyle\sum_{i=1}^{n}(x_{i1}-\overline{x}_{1})(x_{i2}-\overline{x}_{2}) & \displaystyle\sum_{i=1}^{n}(x_{i2}-\overline{x}_{1})^{2} \\ \end{bmatrix}\\\\ (\textbf{W}^{\prime}\textbf{W})^{-1} & = \frac{1}{\det(\textbf{W}^{\prime}\textbf{W})} \begin{bmatrix} n(s_{xx}s_{yy}-s^{2}_{xy}) & 0 & 0 \\ 0 & ns_{xx}s_{yy} & -ns^{2}_{xy} \\ 0 & -ns^{2}_{xy} & ns_{xx}s_{yy} \\ \end{bmatrix} \end{align*}

where $\det(\textbf{W}^{\prime}\textbf{W}) = n(s_{xx}s_{yy} - s^{2}_{xy})$. Once \begin{align*} \textbf{Var}[\textbf{Y}] & = \textbf{Var}[\textbf{Y} - \textbf{W}\beta] = \textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n} \Longrightarrow\\\\ \textbf{Var}[\hat{\beta}] & = \textbf{Var}[(\textbf{W}^{\prime}\textbf{W})^{-1}\textbf{W}^{\prime}\textbf{Y}] = (\textbf{W}^{\prime}\textbf{W})^{-1}\textbf{W}^{\prime}\textbf{Var}[\textbf{Y}]\textbf{W}(\textbf{W}^{\prime}\textbf{W})^{-1}\\\\ & = \sigma^{2}(\textbf{W}^{\prime}\textbf{W})^{-1}\textbf{W}^{\prime}\textbf{W}(\textbf{W}^{\prime}\textbf{W})^{-1} = \sigma^{2}(\textbf{W}^{\prime}\textbf{W})^{-1} \end{align*} However such reasoning leads to the expression \begin{align*} \text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\displaystyle 1 - \frac{s^{2}_{xy}}{s_{xx}s_{yy}}} = \frac{\sigma^{2}}{1 - r^{2}} \end{align*}

Can someone tell me where is the mistake? Thanks in advance!

EDIT

I think I found the mistake: I was miscalculating the inverse of $\textbf{W}^{\prime}\textbf{W}$. Indeed, we have

\begin{align*} (\textbf{W}^{\prime}\textbf{W})^{-1} & = \frac{1}{\det(\textbf{W}^{\prime}\textbf{W})} \begin{bmatrix} s_{xx}s_{yy}-s^{2}_{xy} & 0 & 0 \\ 0 & ns_{yy} & -ns_{xy} \\ 0 & -ns_{xy} & ns_{xx} \\ \end{bmatrix} \end{align*}

\begin{align*} \therefore \text{Var}(\hat{\beta}_{1}) = \frac{n\sigma^{2}s_{yy}}{n(s_{xx}s_{yy} - s^{2}_{xy})} = \frac{\sigma^{2}}{\displaystyle s_{xx} - \frac{s^{2}_{xy}}{s_{yy}}} = \frac{\sigma^{2}}{\displaystyle s_{xx}\left(1 - \frac{s^{2}_{xy}}{s_{xx}s_{yy}}\right)} = \frac{\sigma^{2}}{s_{xx}(1-r^{2})} \end{align*}

I think it is fine now :)

$\endgroup$
  • $\begingroup$ apart from small typos that doesn't affect the final result, yes, it seems you've provided a nearly complete solution :) $\endgroup$ – gunes Apr 2 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.