In Regression and Other Stories, the authors state that heteroscedasticity and non-normal errors are only problematic when predicting from a linear model (1; p. 154-155):
- Equal variance of errors. Unequal error variance (also called heteroscedasticity, in contrast to equal variances, or homoscedasticity) can be an issue when a regression is used for probabilistic prediction, but it does not aﬀect what is typically the most important aspect of a regression model, which is the information that goes into the predictors and how they are combined. If the variance of the regression errors are unequal, estimation is more eﬃciently performed by accounting for this in the model, as with weighted least squares discussed in Section 10.8. In most cases, however, this issue is minor.
- Normality of errors. The distribution of the error term is relevant when predicting individual data points. For the purpose of estimating the regression line (as compared to predicting individual data points), the assumption of normality is typically barely important at all. Thus we do not recommend diagnostics of the normality of regression residuals. For example, many textbooks recommend quantile-quantile (Q-Q) plots, in which the ordered residuals are plotted vs. the corresponding expected values of ordered draws from a normal distribution, with departures of this plot from linearity indicating nonnormality of the error term. There is nothing wrong with making such a plot, and it can be relevant when evaluating the use of the model for predicting individual data points, but we are typically more concerned with the assumptions of validity, representativeness, additivity, linearity, and so on, listed above.
My understanding from the text above is that the authors think violations of these assumptions are not an issue for the estimation of model coefficients. Why are violations of these assumptions only an issue for prediction?
Non-normal errors are discussed in this Cross Validated answer, however I would appreciate an answer that refers more to the underlying mathematics or refers to external sources.
- A. Gelman, J. Hill, A. Vehtari, Regression and Other Stories (Cambridge University Press, 2020) https:/doi.org/10.1017/9781139161879.