1
$\begingroup$

Let $\hat B=(X^{T}X)^{-1}X^{T}Y$ be the least squares estimator of a linear multivariate regression model with response matrix $Y\in\mathbb R^{n\times p}$. Show:

(a) $\mathbb E\left[tr\left((\hat B-B)^{T}(\hat B-B)\right)\right]=tr(\Sigma)tr\left((X^{T}X)^{-1}\right)$

(b) $\mathbb E\left[tr\left((\hat B-B)^{T}\Sigma^{-1}(\hat B-B)\right)\right]=p\cdot tr\left((X^{T}X)^{-1}\right)$

For (a) we know $\mathbb E[\hat B]=(X^{T}X)^{-1}X^{T}\mathbb E[Y]=(X^{T}X)^{-1}X^{T}XB=B$. Hence $\mathbb E[(\hat B-B)(\hat B-B)^T]=Var(\hat B)$.

$\mathbb E\left[tr\left((\hat B-B)^{T}(\hat B-B)\right)\right]=\mathbb E\left[tr\left((\hat B-B)(\hat B-B)^{T}\right)\right]=tr\left(\mathbb E\left[(\hat B-B)(\hat B-B)^{T}\right]\right)=tr\left(\mathbb Var(\hat B)\right)=tr\left(\Sigma \otimes (X^{T}X)^{-1}\right)=tr(\Sigma)tr\left((X^{T}X)^{-1}\right)$

For (b) $\mathbb E\left[tr\left((\hat B-B)^{T}\Sigma^{-1}(\hat B-B)\right)\right]=\mathbb E\left[tr\left(\Sigma^{-1}(\hat B-B)(\hat B-B)^{T}\right)\right]=tr\left(\Sigma^{-1}\mathbb E\left[(\hat B-B)(\hat B-B)^{T}\right]\right)=tr\left( \Sigma^{-1}\mathbb Var\left(\hat B\right)\right)\color{red}{=}tr\left( \Sigma^{-1}\Sigma \otimes (X^{T}X)^{-1}\right)=tr\left(I_p\right)tr\left((X^{T}X)^{-1}\right)=p\cdot tr\left((X^{T}X)^{-1}\right)$

Is that right?

$\endgroup$
  • 1
    $\begingroup$ Is there any particular step you cannot justify? $\endgroup$ – whuber Oct 8 '17 at 21:24
  • 1
    $\begingroup$ step at red marked equality sign..why is $\Sigma^{-1}\left(\Sigma \otimes (X^{T}X)^{-1}\right)=\Sigma^{-1}\Sigma \otimes (X^{T}X)^{-1}$? $\endgroup$ – lemontree Oct 8 '17 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.