# Are $\hat{y} \sim\mathcal{N}(X\beta, \sigma^{2}X(X^{T}X)^{-1}X^{T}$) = $y \sim\mathcal{N}(X\beta, \sigma^{2}I_n$)?

In my lecture note, it states that $$\hat{y} \sim\mathcal{N}(X\beta, \sigma^{2}X(X^{T}X)^{-1}X^{T}$$)

but isn't $$y \sim\mathcal{N}(X\beta, \sigma^{2}I_n$$) ?

Which one is accurate or are they the same thing but with different representation?

## $$\mathbf{y \neq \hat y}$$

• The $$\sigma^2(X^TX)^{-1}$$ is the covariance table $$\text{Cov}(\hat\beta)$$ for the estimated coefficients
• Then $$X Cov(\hat\beta) X^T$$ is the covariance $$\Sigma$$ for the error of the estimated values $$\hat y = X \hat \beta$$

(that is different from the true sample values $$y$$).

## Intuition

Imagine a linear regression line which is always less accurate at the ends due to the uncertainty in the slope.

## Different notation

Possibly it may become more clear when we use different notation (use the $$\hat \mu$$ instead of $$\hat y$$)

• The true mean (conditional on $$X$$) is $$\mu_X = X\beta$$ the $$y$$ and $$\hat y$$ are different derivatives of this.
• The estimated mean $$\hat y$$, or the regression line. You could better interpret $$\hat y$$ as the estimated mean, ie the regression line. Then $$\hat \mu_X \sim \mathcal{N}(\mu, \sigma^2 X (X^TX)^{-1} X^T )$$ is expressing the sample variation of this regression line around $$X\beta$$ (how your estimate of the conditional mean $$\hat \mu_X$$ will vary from experiment to experiment, or sample to sample)
• The sampled data $$y$$. The interpretation of $$y$$ and $$\hat{y}$$ is a bit different. The one is a data point the other is the mean. The sampled data points $$y$$ are distributed around the true mean $$X\beta$$ and, if the variance is homoscedastic and normal distributed then it is $$Y \sim \mathcal{N}(\mu_X,\sigma^2 I_n)$$

In the case of prediction, if you would like to estimate an error for the estimate of a new value, then you would actually use the sum of the two variances expressed above (the estimate of the mean plus the 'error' of a sampled value with respect to the mean).

For matrices $A$ and $B$, $(AB)^{-1} = B^{-1}A^{-1}$. Use this on your first formula and see what you get after simplification.

• This only works for square matrices. Which is actually interesting. When $X$ is square then $y = \hat y$ – Sextus Empiricus Oct 2 '18 at 7:58