Let $\hat\beta$ be the least squares estimator of $\beta$ in the regression model $y = x \beta + \mu$ where x is an $n \times k$ matrix with rank k and $\mu \tilde ~ N(0, \sigma^2 I)$.

1) Find the non-random matrices $A$ and $B$, such that $\hat\beta$ - $\beta$ can be written in the form $A\mu$ and the sum of the squared residuals can be written as $\mu'B\mu$.

2) Show that $B$ is idempotent and that $AB=0$

3) Indicate how these results enable one to obtain the distribution of

\begin{align} \frac{ (\hat\beta - \beta)'X'X(\hat\beta - \beta) }{(Y-X\hat\beta)'(Y-X\hat\beta)}. \frac{ n-k }{k}. \end{align}

I think I can handle the second part (2), but I have a problem getting the non-random matrices and part 3

  • 3
    $\begingroup$ Please explain what you have tried so far. $\endgroup$ – Christoph Hanck Oct 14 '16 at 8:41
  • 1
    $\begingroup$ Please add the self-study tag as well. $\endgroup$ – Xi'an Oct 14 '16 at 8:59
  • $\begingroup$ Hints: it is not true that $A$ is square as $\mu$ is $n\times 1$ and $\beta$ is $k\times 1$, so $A$ must be $k\times n$. Take the expression for the OLS estimator and substitute for $y$. $\endgroup$ – Christoph Hanck Oct 14 '16 at 10:57

(Since this is tagged as self-study, just some tips without details.)

  1. Note that the solution of OLS is

    $\hat{\beta} = \frac{X^Ty}{X^TX}$. (1)

    In addtion, it's given that

    $y = X \beta + \mu$. (2)

    Now insert (2) into (1), subtract $\beta$, and show that all terms cancel out, except those multiplying $\mu$. This should give you the $A$ solving $\hat{\beta} - \beta = A\mu$.

    For $B$, see the part in the Wikipedia entry discussing estimation, in particular the annihilator matrix $M$.

  2. Once you solve $A$ and $B$, this should be easy. As a hint, note that in the same Wikipedia entry, the projection matrix $P$ satisfies $P^2 = P$.

  3. In $\frac{ (\hat\beta - \beta)^TX^TX(\hat\beta - \beta) }{(Y-X\hat\beta)^T(Y-X\hat\beta)}$,

    substitute the expression you found for $\hat\beta - \beta$, and consider how the denominator relates to the residual sum of squares. After some substitutions, use known properties of the normal distribution.

| cite | improve this answer | |
  • $\begingroup$ According to the guidelines, I have calculated matrices $A$ and $B$. Check in the comment if they are correct. $\endgroup$ – Alexxio Oct 14 '16 at 20:24
  • $\begingroup$ @Alexxio No, both are wrong. For the first, note that $\hat{\beta} - \beta = \frac{X^T(X\beta + \mu)}{X^TX} - \beta $, and continue from here. For the second, try to see why it is the square of the annihilator matrix. $\endgroup$ – Ami Tavory Oct 14 '16 at 22:38
  • $\begingroup$ @Alexxio $\hat\beta - \beta = \frac{X^T(X\beta + \mu)}{X^TX} - \beta = \frac{X^TX\beta + X^T\mu - X^tX \beta}{X^TX} = \frac{X^T \mu}{X^T X}$. For the annihilator matrix, read the Wikipedia entry. $\endgroup$ – Ami Tavory Oct 15 '16 at 6:29
  • $\begingroup$ ok so in this case, A = $\frac{x^T}{x^Tx}$ , right? $\endgroup$ – Alexxio Oct 15 '16 at 6:55
  • $\begingroup$ That is correct. Please now read the wiki in the link, and see why $B$ is the square of the annihilator matrix. $\endgroup$ – Ami Tavory Oct 15 '16 at 7:07

1) The solution of OLS is

$\hat{\beta} = \frac{X^Ty}{X^TX}$. (1)

We have:

$y = X \beta + \mu$. (2)

Therefore, $\hat{\beta} = \frac{\beta + \mu}{X^TX}$. = $\frac{\beta + \mu}{X^TX}$

= $\beta - \hat\beta = x^{-1}\mu$

hence A = $x^{-1}$

2) $\beta = (y - x)^T(y - xb)$

= $(x\beta + \mu - xb)^T(x\beta + \mu - xb)$

=$\mu'\mu = \mu'B\mu$

=$B=1$ (Am not sure about this)

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.