So I have this linear regression model shown below and I'm supposed to be showing that equation 3 is equal to equation 4. There's a hint that says a 2x2 inverse matrix appears in the proof, but the inverse of xx' doesn't exist (?)

Am I supposed to be rewriting beta in matrix terms? I'm not sure what I'm supposed to be doing here.

enter image description here


As the first step, we will rewrite the Equation 3 more explicitly. The first term $E[X'X]$ can be rewritten as:

Note: The random vector $X$ is composed of a constant value of 1 due to interception and a random term $X_1$. Since this is the only random variable in the vector, I will use $X$ instead of it.

\begin{equation*} E[X'X]^{-1}=E\left\{ \begin{bmatrix} 1 & X \\ X & X^2 \end{bmatrix}\right\}^{-1}= \begin{bmatrix} 1 & E[X] \\ E[X] & E[X^2] \end{bmatrix}^{-1}\\[2em]= \frac{1}{E[X^2]-E[X]^2} \begin{bmatrix} E[X^2] & -E[X] \\ -E[X] & 1 \end{bmatrix}\\[2em]=\frac{1}{Var(X)} \begin{bmatrix} E[X^2] & -E[X] \\ -E[X] & 1 \end{bmatrix} \end{equation*}

Then, the second term $E[XY]$ can be written as:

\begin{equation*} E[XY]= E\left\{ \begin{bmatrix} 1 \\ X \end{bmatrix}Y\right\}=\begin{bmatrix} E[Y] \\ E[XY] \end{bmatrix} \end{equation*}

Now, we can substitute these matrices into Equation 3. With some algebra, we can easily reach Equation 4.

\begin{equation*} \beta=E[X'X]^{-1}E[XY]= \frac{1}{Var(X)} \begin{bmatrix} E[X^2] & -E[X] \\ -E[X] & 1 \end{bmatrix}\begin{bmatrix} E[Y] \\ E[XY] \end{bmatrix}\\[2em]=\frac{1}{Var(X)}\begin{bmatrix} E[X^2]E[Y]-E[X]E[X,Y] \\ E[XY]-E[X]E[Y] \end{bmatrix}\\[2em]=\frac{1}{Var(X)}\begin{bmatrix} E[X^2]E[Y]-E[Y]E[X]^2+E[Y]E[X]^2-E[X]E[X,Y] \\ Cov(X,Y) \end{bmatrix}\\[2em]=\frac{1}{Var(X)}\begin{bmatrix} E[Y]Var(X)-E[Y]Cov(X,Y) \\ Cov(X,Y) \end{bmatrix}=\begin{bmatrix} E[Y]-E[X]\frac{Cov(X,Y)}{Var(X)} \\ \frac{Cov(X,Y)}{Var(X)} \end{bmatrix}=\begin{bmatrix} \beta_0 \\ \beta_1 \end{bmatrix} \end{equation*}

  • $\begingroup$ Wow, thank you so much for the quick reply. I really appreciate it. I was just stuck at finding the inverse properly, so I couldn't progress any further. I understand your working. Thanks a lot! $\endgroup$ – Elisia Hearts May 2 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.