Justifying Residual Histograms and QQ Plots for Linear Regression

Question

Conceptually, I am having a hard time as to why we consider the quantile-quantile plot for linear regression diagonistics, and I cannot seem to get a clear answer after searching extensively.

The common and superficial answer I find is that comparing our residuals to a normal distribution with a QQ plot (or by constructing a residual histogram) ensures that we satisfy the assumption that our errors are normally distributed for each x-value. But therein lies a flaw. Namely, QQ plots and histograms of residuals combine the errors from all x-values into one distribution. In this way, we are no longer considering the distributions at each x value and instead are looking at a single "collage" distribution. As such, even if our QQ plot looks linear or our histogram plot looks approximately normal, we cannot be sure that the individual distributions at each x value do. Is there some sort of conceptual justification for this?

Answer 1

You have a good question. An analogy I find helpful is that residual diagnostics are like health or machine checks. The absence of an apparent problem is not itself proof that one doesn't exist, yet there is an optimistic undercurrent that no news is good news.

The normal quantile plot of residuals has indirect value as well as direct value, and in my experience the indirect value often exceeds the direct value.

Graphs rarely provide cast-iron or unproblematic evidence here, but they often yield helpful indications.

Specifically, normally distributed errors are the least important ideal condition for regression (often unfortunately explained as an assumption).

A normal quantile plot can be valuable generally, just as a display of the distribution of residuals that can show up other features, such as skewness, spikes, gaps, outliers, and so forth, that may be unsurprising or surprising. Either way, they may point to problems that need thought or even action.

Outliers won't always be obvious on such a plot any way: they may exert so much leverage that the fitted regression passes very near and the corresponding residual is very small. No single plot is necessary or sufficient!

The use of a normal scale can just be a conventional reference.

That applies to histograms with a normal scale superimposed, which I don't use nearly as much as normal quantile plots.

Your other comments are interesting. A simple but powerful principle is that practical success in statistics grows out of lengthy experience, including lessons from bad or naive decisions, as well as out of understanding the principles of what is being done. How could it be otherwise? For example, I have seen (and sympathised with) the frustration of students and colleagues when I've noted (say) that a logarithmic transformation will help mightily (or sometimes not at all) and they wonder how they are supposed to know that from what they've read. The answer is often that the advice arises from experience with similar data (which they don't yet have). Conversely, I am embarrassed by many analyses and even some publications from earlier in my career.

Answer 2

We only justify the "distribution" as a whole.

Whether "each" data point follows that distribution is mainly determined by the sampling process, i.e., how we draw data from the parent distribution. But if we see overt outliers, we may question if that was sampled from the correct parent distribution.

• If the person who conducted the sampling/data collection confirms it, then we know our assumption on the parent population/distribution is incorrect.
• If errors during sampling/data collection indeed occurred, then we need to remove all data from the wrong population, irrespective of whether they look strange in any diagnostic plots.

To sum up, the diagnostic plots provide information about the population/distribution, but can at most provide some hints on whether some data points are correctly sampled, which has to be answered by the data collector.