# Justifying Residual Histograms and QQ Plots for Linear Regression

Conceptually, I am having a hard time as to why we consider the quantile-quantile plot for linear regression diagonistics, and I cannot seem to get a clear answer after searching extensively.

The common and superficial answer I find is that comparing our residuals to a normal distribution with a QQ plot (or by constructing a residual histogram) ensures that we satisfy the assumption that our errors are normally distributed for each x-value. But therein lies a flaw. Namely, QQ plots and histograms of residuals combine the errors from all x-values into one distribution. In this way, we are no longer considering the distributions at each x value and instead are looking at a single "collage" distribution. As such, even if our QQ plot looks linear or our histogram plot looks approximately normal, we cannot be sure that the individual distributions at each x value do. Is there some sort of conceptual justification for this?

• You hit the nail right on the head! Residual diagnostics are fraught with problems, for exactly the reasons you mention. Specifically, the assumptions concern the conditional distributions of residuals, whereas these plots only examine the marginal distributions. And as such, they can be greatly misleading. For example, with a highly discrete Y, these plots can fool you into thinking that normality is a "reasonable assumption." Commented Jul 20 at 18:55

You have a good question. An analogy I find helpful is that residual diagnostics are like health or machine checks. The absence of an apparent problem is not itself proof that one doesn't exist, yet there is an optimistic undercurrent that no news is good news.

The normal quantile plot of residuals has indirect value as well as direct value, and in my experience the indirect value often exceeds the direct value.

Graphs rarely provide cast-iron or unproblematic evidence here, but they often yield helpful indications.

Specifically, normally distributed errors are the least important ideal condition for regression (often unfortunately explained as an assumption).

A normal quantile plot can be valuable generally, just as a display of the distribution of residuals that can show up other features, such as skewness, spikes, gaps, outliers, and so forth, that may be unsurprising or surprising. Either way, they may point to problems that need thought or even action.

Outliers won't always be obvious on such a plot any way: they may exert so much leverage that the fitted regression passes very near and the corresponding residual is very small. No single plot is necessary or sufficient!

The use of a normal scale can just be a conventional reference.

That applies to histograms with a normal scale superimposed, which I don't use nearly as much as normal quantile plots.

Your other comments are interesting. A simple but powerful principle is that practical success in statistics grows out of lengthy experience, including lessons from bad or naive decisions, as well as out of understanding the principles of what is being done. How could it be otherwise? For example, I have seen (and sympathised with) the frustration of students and colleagues when I've noted (say) that a logarithmic transformation will help mightily (or sometimes not at all) and they wonder how they are supposed to know that from what they've read. The answer is often that the advice arises from experience with similar data (which they don't yet have). Conversely, I am embarrassed by many analyses and even some publications from earlier in my career.

We only justify the "distribution" as a whole.

Whether "each" data point follows that distribution is mainly determined by the sampling process, i.e., how we draw data from the parent distribution. But if we see overt outliers, we may question if that was sampled from the correct parent distribution.

• If the person who conducted the sampling/data collection confirms it, then we know our assumption on the parent population/distribution is incorrect.
• If errors during sampling/data collection indeed occurred, then we need to remove all data from the wrong population, irrespective of whether they look strange in any diagnostic plots.

To sum up, the diagnostic plots provide information about the population/distribution, but can at most provide some hints on whether some data points are correctly sampled, which has to be answered by the data collector.

• I am still struggling to understand your answer. Let me try to recapitulate it: when using a QQ plot or residual histogram, we attempt to verify that the collection of all errors represents a normal distribution. From there, if we conduct a sound sampling process, we may conclude that the error at each x-value is also normally distributed. Commented Jul 14 at 20:40
• I think that's the basic idea, although there are a lot of less important nuisances, e.g., (1) many are reluctant to conclude $\epsilon$ follow $N(\mu,\sigma^2)$, but will only say "we don't have enough evidence indicating that $\epsilon$'s are not normal"; (2) even if our model on the mean ($X\beta$) is good, residuals are still just proxies of the unobservable true errors and residuals are dependent to some little degree; and (3) if our mean model is not a good one, residuals and true errors can be wildly different and we cannot say much about errors by using residuals. Commented Jul 14 at 21:13
• Besides that, normality is a fairly unimportant assumption when we have decent sample size. It hardly practically affects our inference on means. Commented Jul 14 at 21:23
• May I ask, how does one become so entrenched and well-versed in statistics? I feel as though I am routinely getting tripped up by assumptions and trying to make sense of concepts that seem so obscure. I feel as though I am in this constant recursive rabbit hole, where I try to understand a new concept in an effort to better understand another concept and find myself completely confused by the concept that was supposed to enlighten me. Commented Jul 15 at 0:21
• I guess statistics students generally come with two types of backgrounds: applied sciences, or mathematical sciences. They face very different issues when starting to learn statistics. But they will gradually merge together once their weakness part is strengthened. With that said, I would suggest reading at least one good book in each of the two broad areas, applied data analysis and mathematical statistics. The road will be easier once we can walk with both legs. Typically, university courses will adopt a well-accepted book, and their syllabi might include book recommendations. Commented Jul 15 at 0:49