# Standardized residuals vs. regular residuals

I've got an easy question concerning residual analysis. So when I compute a QQ-Plot with standardized residuals $\widehat{d}$ on the y-axis and I observe normal distributed standardized residuals, why can I assume that the error term $u$ is normal distributed? I'd think that if $\widehat{d}$ looks normal distributed I just could assume that the standardized error term $d$ should be normal distributed.

So why can we assume that $u\sim N$ when we just observe that $\widehat{d}\approx N$. By the way can we assume it?

• could you please mention how you calculate $u$ and $\hat d$ – suncoolsu Jul 12 '11 at 16:41
• Hi. you can't calculate $u$ (except you're observing the whole population). So I'd say I calculate $\widehat{u}=y-\widehat{y}$ and $\widehat{d}=\widehat{u}/\widehat{\sigma^2}$. I guess I've got an idea of the answer, but I can not show it. What I know is, that we often assume that $u\sim N(0,\sigma^2)$, so when we divide $u$ by $\sigma^2$ I assume that we divide the first 2 moments of the distribution as well $u/\sigma^2 \sim N(0/\sigma^2,\sigma^2/\sigma^2)$ where $u/\sigma^2=d$ ist the std.error term which is (from above) $d\sim N(0,1)$. But I'm not sure if this makes sense (That's why I ask) – MarkDollar Jul 12 '11 at 19:46
• ... Thus when $u\sim N(0,\sigma^2)$ we assume automatically that $d\sim N(0,1)$. Is this correct? And where can I find a proof that this is correct? – MarkDollar Jul 13 '11 at 8:09
• In practical situations we estimate $\sigma^2$, then $\hat d$ is no longer gaussian. Instead, it is a $t$. But a $t$ with higher than 10 is not very different from a gaussian distribution. – suncoolsu Jul 13 '11 at 14:31
• @suncoolsu: I absolutely don't understand what you mean :) Please clarify and give a reference. Thank you! – MarkDollar Jul 13 '11 at 17:27

The $u$s are unobserved and the $\hat{d}$s are just estimates of them.

This inference is no different from any other inference we make. We assume a default (you could call it a 'null'). In this case, it's that the underlying distribution is Gaussian. We examine the data to see if they are inconsistent with our default hypothesis. If the qq-plot of our residuals looks sufficiently Gaussian for your satisfaction, then we stick with that assumption. In truth, no matter how non-normal our residuals appear, they could still have come from an underlying Gaussian distribution, but at some point, we just don't believe it anymore. Another way to phrase this is that we don't assume they're Gaussian because the qq-plot looks Gaussian, rather we don't stop assuming they're Gaussian because the qq-plot doen't look sufficiently non-Gaussian.

Some people have trouble with this line of reasoning; which is perfectly fine. You might be interested in checking out the Bayesian approach to statistics.

In a linear model $$y = X\beta + u$$ with $$u \sim N(0, \sigma^2I)$$, the vector of raw residuals is $$\hat u = y - \hat y = (I - H)y$$ where the hat matrix $$H = X(X'X)^{-1}X'$$. The response $$y$$ is normally distributed given the assumed normality of the error terms. Consequently, if the model assumes normality correctly, $$\hat u$$ is also normally distributed since each residual is a linear combination of $$y$$. Therefore, if a QQ-plot does not support the normality of $$\hat u$$, it warrants a question about the normality of $$y$$, which in turn raises a doubt about the normality assumption of $$u$$. On the other hand, if there is a lack of evidence to reject the normality of $$\hat u$$, the logic is to accept the normality of $$u$$.

On the specific question about checking normality of $$\hat d = \hat u/\hat \sigma$$ and its relationship to the normality of $$u$$ ...

As just mentioned, $$\hat u$$ is normally distributed under the model, so is $$\hat u/c$$ for any non-zero constant $$c$$, such as $$\sigma$$. If $$\sigma$$ were known, the scaled residual $$\hat u/\sigma$$ would be better than $$\hat u$$ for identifying potential outliers. As $$\sigma$$ is usually unknown, $$\hat d$$ = $$\hat u/\hat \sigma$$ is used instead. However, the approach of checking the normality of $$\hat d$$ to validate the normality of $$u$$ is approximate and less preferred given that its divisor $$\hat \sigma$$ is a statistic, not a constant.

Is then using $$\hat u$$ to assess the normality of $$u$$ a perfect approach? Not necessarily...

Note that $$Var(\hat u)$$ = $$\sigma^2(I - H)$$, indicating that raw residuals are correlated and variances are not constant. More specifically, such residuals are not sample points randomly (independently) drawn from a common underlined distribution. Is it then appropriate to subject the residuals to a univariate QQ-plot to assess normality?

In brief, any conclusion about normality from a QQ-plot is visual based, thus subjective, regardless of which type of residuals (raw or otherwise) is evaluated. Besides, normality assessment is only one aspect of model adequacy assessment. While raw residuals may be more appropriate with QQ-plot, other types of residuals may be better suited with other diagnostic tools.