0
$\begingroup$

As the question above asks. I am attempting to obtain the variance covariance matrix of the fitted values $\hat{Y}_{i}$ in terms of the hat matrix. I essentially have completed it, but there is one step I don't understand (or maybe remember the property). Also to mention I'm doing this in matrix notation.

Recall: $\hat{Y} = \mathbf{HY}$

Applying properties of derivations for a varaince-covariance matrix:

$$\Sigma_{\hat{Y}\hat{Y}} = \mathbf{H} \mathbf{\Sigma_{YY}} \mathbf{H}^{t}$$ Where $\mathbf{H}^{t}$ is the transpose.

A solution for the problem proceeds as follows:

$$= \mathbf{H}\sigma^{2}\mathbf{I}\mathbf{H}^{t} \\ = \mathbf{H}\sigma^{2}\mathbf{I}\mathbf{H}\ \text{since H is symmetric} \\ = \sigma^{2}\mathbf{HH} \\ = \sigma^{2}\mathbf{H}$$

My only issue with any of this is treating $\sigma^{2}$ as a scalar. For me to be able to write $\mathbf{\Sigma_{YY}} = \sigma^{2}\mathbf{I}$, I have to know that there are no off diagonal terms with covariance not equal to $0$. How do I know this here? I must be missing something in regards to the theory which would allow me to express the variance-covariance matrix in this way.

$\endgroup$
3
  • 1
    $\begingroup$ This is what iid errors means. Weaker assumptions, such as homoscedastic errors, implies the same thing. $\endgroup$
    – whuber
    Commented May 20, 2021 at 1:45
  • $\begingroup$ If I didn't have the assumption of homoscedasticity, then I couldn't pull that scalar out right? What would happen in such a scenario? Well I know there a bunch of remedial measures I suppose, so I guess for now in my learning process that will have to do. $\endgroup$ Commented May 20, 2021 at 2:04
  • 1
    $\begingroup$ If you're not assuming homoscedasticity, then you are doing generalized least squares. The concepts are identical but a little more linear algebra is involved and, indeed, $\sigma^2$ is replaced by an unknown scalar multiple of the correlation matrix you posit. $\endgroup$
    – whuber
    Commented May 20, 2021 at 2:06

1 Answer 1

1
$\begingroup$

Here is this deduction using perhaps more instructive notation,

$$cov(\hat Y)=cov(X\hat\beta)=X \cdot cov(\beta)X^T=\sigma^2 X(X^TX)^{-1}X=\sigma^2 H$$

Where the key step is,

$$cov(\beta)=\sigma^2 (X^TX)^{-1}$$

Which is true under homoskedasticity. Essentially we are saying that,

$$[Cov(\hat Y)]_{ij} = \sigma^2 \cdot h_{ij}$$

Where $h_{ij}=[X(X^TX)^{-1}X^T]_{ij}=I$.

$\endgroup$
1
  • $\begingroup$ Ah yes..THanks for the reminder. $\endgroup$ Commented May 20, 2021 at 2:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.