# Residuals in linear regression

Suppose I have a linear regression model $$\mathbf y = \mathbf X \mathbf b + \boldsymbol{\varepsilon},\,\boldsymbol{\varepsilon}\sim \mathcal N(\mathbf 0, \sigma ^2\mathbf I)$$ and use ML approach in order to estimate $\mathbf b$, that is: $$\hat{\mathbf b}_{\mathrm {ML}} = (\mathbf X ^\top \mathbf X)^{-1}\mathbf X^\top \mathbf y$$ Consider residuals: $$\mathbf r = \mathbf y - \mathbf X\hat{\mathbf b}_{\mathrm {ML}} = (\mathbf I - \mathbf X(\mathbf X ^\top \mathbf X)^{-1}\mathbf X^\top) \mathbf y \sim \mathcal N(\mathbf 0, \sigma ^2(\mathbf I - \mathbf P)^2),$$ where $\mathbf P = \mathbf X(\mathbf X ^\top \mathbf X)^{-1}\mathbf X^\top$.

How can I test normality of $\mathbf r$ using QQ- or PP-plots?

And another question: what is an unbiased estimate for $\sigma$ (since ML approach leads to biased estimate for $\sigma ^2$ as well as $\sigma$)?

• As in one of the recent posts, also here I think it would be more common to use either $\hat \beta$ or $\text{b}$ in place of $\hat{\text{b}}$. Commented Sep 25, 2015 at 14:56
• I use C. Bishop's notation (except $\mathbf b$, while Bishop uses $\mathbf w$) for which $\beta$ stands for precision, i.e. inverse variance. However, thanks for advice, I will keep it in mind. Commented Sep 25, 2015 at 15:31
• How are you going to calculate $(I-P)^{-1}$? Commented Sep 27, 2015 at 23:36
• Oops, I have missed the fact that $\mathbf P$ is a projector, that is its inverse does not exist. Therefore, how can I test normality of $\mathbf r$ using QQ-plot? Commented Sep 28, 2015 at 12:21