# Does regression work on data that isn't normally distributed?

I'm trying to see if variables x and y together or separately significantly affect Q_7 (the histogram for which is above). I've run a Shapiro-Wilk normality test and got the following

shapiro.test(Q_7)
## data:  Q_7
## W = 0.68439, p-value < 2.2e-16

With this distribution, will the following regression work? Or is there another test I should be doing?

lm(Q_7 ~ x*y)
• check residuals, not data Nov 26, 2016 at 12:52
• Try log-transforming Q_7. At the moment it is heavily right-skewed. Check the distributions of the predictors as well.
– Joe
Nov 26, 2016 at 13:16
• Look up the Gauss Markov Theorem. Nov 26, 2016 at 13:25
• Try with the square root transformation. If you have many zeros the log transformation may not work well. Also, since you are dealing with counts, Poisson negative binomial regression are more natural choices. Nov 26, 2016 at 13:59
• What does "non data" mean? Nov 26, 2016 at 18:46

A regression analysis assumes that the data is normally distributed conditioned on the variables in the regression model. That is, if this is the regression model: $$y=X\beta+\varepsilon$$ where $X$ is your matrix of regressor variables, $y$ is the (vector of) data to be explained, $\beta$ is a vector of coefficients on the regressors and $\varepsilon$ is random variability (typically considered noise), then the assumption of Normality applies strictly to $\varepsilon$, not to $y$ (edit: well, strictly speaking it applies to the conditional distribution $y|X$ (which is the same as the distribution of $\varepsilon$), but not to the marginal distribution of $y$). In other words, the data should be Normally distributed once the effects of the regressors have been accounted for, but not (necessarily) before.

What you're testing here is the distribution of $y$, where what you want to test is the distribution of $\varepsilon$. Of course you don't know $\varepsilon$, but you can estimate it by running the regression and examining the distrbution of the residuals $\hat\varepsilon=y-X\hat\beta$ (where $\hat\beta$ are the estimated coefficents from the regression). These residuals $\hat\varepsilon$ are an estimate of $\varepsilon$, and so their distribution will be an approximation of the distribution of $\varepsilon$.

• This is a good brisk summary of standard stuff but seems to miss a key feature of this question, which is that with a skewed but non-zero response this functional form is unlikely to be a good idea. To avoid negative predictions, and on other grounds, Poisson regression seems a better starting point. Aug 22, 2017 at 12:00

First of all (as is pointed out by Ruben van Bergen), the distribution of $y$ (or $X$, for that matter) is not relevant. If you were to make a distributional assumption, it would be on your residuals $\varepsilon$, so that is what you should check.

But more importantly, you don't need the normality assumption at all for your estimation to work. You are using R's lm function, which estimates your model using ordinary least squares (OLS). That method will give you a correct estimation of the expectation of $Y$ conditional on $X$ as long as:

• $\mathbb{E}[\varepsilon|X] = 0$ (there is no external factor affecting both your outcome and your explanatory variables).
• $\mathrm{Var}(\varepsilon) < \infty$ (your residuals have finite variance).

If you further make the assumption that you residuals are uncorrelated and that they all have the same variance, then the Gauss-Markov theorem applies and the OLS is the best linear unbiased estimator (BLUE).

If your residuals are correlated or have different variances, then OLS still works but it can be less precise, which must be reflected in the way you report the confidence intervals of your estimates (using, say robust standard errors).

If you also make the assumption that your residuals are normally distributed, then OLS becomes asymptotically efficient because it is equivalent to maximum likelihood.

So the regression may work better if your data are normally distributed, but it will still work if they aren't.