# Assumptions of linear models and what to do if the residuals are not normally distributed

I am a little bit confused on what the assumptions of linear regression are.

So far I checked whether:

• all of the explanatory variables correlated linearly with the response variable. (This was the case)
• there was any collinearity among the explanatory variables. (there was little collinearity).
• the Cook's distances of the datapoints of my model are below 1 (this is the case, all distances are below 0.4, so no influence points).
• the residuals are normally distributed. (this may not be the case)

But I then read the following:

violations of normality often arise either because (a) the distributions of the dependent and/or independent variables are themselves significantly non-normal, and/or (b) the linearity assumption is violated.

Question 1 This makes it sound as if the independent and depend variables need to be normally distributed, but as far as I know this is not the case. My dependent variable as well as one of my independent variables are not normally distributed. Should they be?

Question 2 My QQnormal plot of the residuals look like this:

That slightly differs from a normal distribution and the shapiro.test also rejects the null hypothesis that the residuals are from a normal distribution:

> shapiro.test(residuals(lmresult))
W = 0.9171, p-value = 3.618e-06

The residuals vs fitted values look like:

What can I do if my residuals are not normally distributed? Does it mean the linear model is entirely useless?

• Your residuals versus fitted plot suggests that your dependent variable has a lower bound. This could drive the patterns you see. This could give you an indications for alternative models you could consider. Commented May 28, 2014 at 19:21

First off, I would get yourself a copy of this classic and approachable article and read it: Anscombe, F. J. (1973) Graphs in statistical analysis. The American Statistician. 27:17–21.

Answer 1: Neither the dependent nor independent variable needs to be normally distributed. In fact they can have all kinds of loopy distributions. The normality assumption applies to the distribution of the errors ($$Y_{i} - \widehat{Y}_{i}$$).

1. One is the assumption of linearity. This means that the trend in $$\overline{Y}$$ across $$X$$ is expressed by a straight line (Right? Straight back to algebra: $$y = a +bx$$, where $$a$$ is the $$y$$-intercept, and $$b$$ is the slope of the line.) A violation of this assumption simply means that the relationship is not well described by a straight line (e.g., $$\overline{Y}$$ is a sinusoidal function of $$X$$, or a quadratic function, or even a straight line that changes slope at some point). My own preferred two-step approach to address non-linearity is to (1) perform some kind of non-parametric smoothing regression to suggest specific nonlinear functional relationships between $$Y$$ and $$X$$ (e.g., using LOWESS, or GAMs, etc.), and (2) to specify a functional relationship using either a multiple regression that includes nonlinearities in $$X$$, (e.g., $$Y \sim X + X^{2}$$), or a nonlinear least squares regression model that includes nonlinearities in parameters of $$X$$ (e.g., $$Y \sim X + \max{(X-\theta,0)}$$, where $$\theta$$ represents the point where the regression line of $$\overline{Y}$$ on $$X$$ changes slope).

2. Another is the assumption of normally distributed residuals. Sometimes one can validly get away with non-normal residuals in an OLS context; see for example, Lumley T, Emerson S. (2002) The Importance of the Normality Assumption in Large Public Health Data Sets. Annual Review of Public Health. 23:151–69. Sometimes, one cannot (again, see the Anscombe article).

However, I would recommend thinking about the assumptions in OLS not so much as desired properties of your data, but rather as interesting points of departure for describing nature. After all, most of what we care about in the world is more interesting than $$y$$-intercept and slope. Creatively violating OLS assumptions (with the appropriate methods) allows us to ask and answer more interesting questions.

• Thanks! In the slides of some statistics course it says that if the assumptions fail you can try to transform Y or transform the explanatory variables. When I transform the Y by doing for example lm(Y^0.3~+X1+X2+...) then my residuals do become normally distributed. Is this a valid thing to do? Commented May 27, 2014 at 18:16
• @Stefan Yes! Transforming a response is often a good thing to do, log, and simple power transforms are common. Commented May 27, 2014 at 22:00
• @Stefan Maybe, maybe not. If you transform your outcome, then your inferences based on the transformed relationships do not necessarily apply to the inverse transformations after you have performed your analysis; this is because $\text{Var}(f(x) \ne f(\text{Var}(x))$. So if you analyze $\ln Y =\beta_{0} + \beta_{X}X + \varepsilon$, finding a significant $\beta_{X}$ does not necessarily translate into a significant $e^{\beta_{X}}$, nor does CI$\beta_{X}$ necessarily correspond to $e^{\text{CI}\beta_{X}}$. Commented May 27, 2014 at 22:18
• @stackoverflowuser2010 Because they do not know what they are talking about? The assumption is built right into the mathematical formalism: $Y = \beta_{0} + \beta_{X}X + \varepsilon$ where $\varepsilon \sim \mathcal{N}(0,\sigma)$. Note that last part: it's the residuals not the variables that are distributed normally. Look: (1) simulate X using a uniform distribution from, oh, say 0 to 100; (2) simulate $Y = 3 + 0.5\times X + \mathcal{N}(0,1)$; (3) regress $Y$ on $X$ and recover $\beta_{0}\approx 3, \beta_{X}\approx 0.5$. Then look at the histograms of $X$ and $Y$. Commented Mar 8, 2016 at 0:20
• The last paragraph of your answer, @Alexis, was very eye-opening. Thanks for that! Commented May 15, 2021 at 3:32

• in spite of your assurances, the residual plot shows that the conditional expected response isn't linear in the fitted values; the model for the mean is wrong.

• you don't have constant variance. The model for the variance is wrong.

you can't even assess normality with those problems there.

• Please elaborate on how you've concluded about linearity by looking at the plots? I understand that the homoskedasticity assumption is not met here. Commented May 18, 2019 at 15:01
• The conditional mean of residuals is changing as $\hat{y}$ changes; there's a clear down trend then a distinct jump up as we move right. If you can't see it, cut the plot into say 4 slices. I'd put the middle of the range of predicted values about $\hat{y}=30$, so cut it there, and then cut each half in half - say at $0$ and $60$. Now looking at the points within each of those slices ( $<0$,$0-30$, $30-60$, $>60$), draw your best estimate of a straight line. For me the middle two are nearly coincident, so I combined their lines, giving something like this Commented May 19, 2019 at 5:40
• In the middle half, nearly all the residuals are negative, in the outer parts nearly all the residuals are positive. These are not how random residuals look. Commented May 19, 2019 at 5:44
• Thanks, @Glen_b. After a long gap, I'm revisiting my concepts so couldn't visualize at first place. Commented May 19, 2019 at 8:46
• While there's not much to go on here, I expect the original data are non-negative, and either a generalized linear model (perhaps a gamma with log-link) or a transformation (likely a log-transformation) would be a more suitable choice. Commented May 23, 2019 at 8:11

1. Sometimes non-normality of residuals indicates the presence of outliers. If this is the case, handle the outliers first.

2. Maybe using some transformations solve the purpose however, it has consequences. Like the interpretation of coefficients changes if we transform variables.

3. Additionally, to deal with multi-colinearity, you can refer https://www.researchgate.net/post/My_data_has_the_problem_of_multicolinearity_Removing_unique_variables_using_variance_inflation_factor_VIF_didnt_work_Any_solution

• I find your first point interesting! Any articles/blogs/videos/referances you could point out regarding this? I ask this because I am working on a regression problem and most of the numerical features in my data are right skewed and contain outliers. I think it might be the reason I am not getting normal distribution in residual plot. Also my learning curves are not making any sense so I would like to get more info on this topic! Commented Jan 8, 2022 at 18:15

The most accessible exploration of the impact of non-normal errors that I have found is this paper by Schmidt and Finan.

Here is the summary of the results in the abstract:

Although outcome transformations bias point estimates, violations of the normality assumption in linear regression analyses do not. The normality assumption is necessary to unbiasedly estimate standard errors, and hence confidence intervals and P-values. However, in large sample sizes (e.g., where the number of observations per variable is >10) violations of this normality assumption often do not noticeably impact results. Contrary to this, assumptions on, the parametric model, absence of extreme observations, homoscedasticity, and independency of the errors, remain influential even in large sample size settings.

I wouldn't say the linear model is completely useless. However, this means that your model doesn't correctly/fully explain your data. There is a part where you have to decide whether the model is "good enough" or not.

For your first question, I don't think that a linear regression model assumes that your dependent and independent variables have to be normal. However, there is an assumption about the normality of the residuals.

For your second question, there is two different things you could consider :

1. Check different kind of models. Another model might be better to explain your data (for example, non-linear regression, etc). You would still have to check that the assumptions of this "new model" are not violated.
2. Your data may not contain enough covariates (dependent variables) to explain the response (outcome). In this case, you cannot do anything else. Sometimes, we may accept to check if the residuals follow a different distributions (e.g. t-distribution) but it doesn't seem to be the case for you.

In addition to your question, I see that your QQPlot is not "normalized". Usually it is easier to look at the plot when your residuals are standardised, see stdres.

stdres(lmobject)

I hope it helps you, maybe someone else will explain this better than me.