# Impact of regression normality-assumption on model comparison & prediction?

This question is a continuation of the discussion here:

How to test the statistical significance for categorical variable in linear regression?

Following Macro's suggestion, I started a new thread.

The new question is not limited to only study the inclusion/exclusion of categorical variable.

It's about general model comparison and prediction.

I found that my data is highly non-normal. The QQ plot is as follows: the curve is all below the straight 45 degree line. The curve is tangent to that straight line. And the curve looks like the curve of f(x)=-x^2 ( shape-wise).

It's not the entire set of points that's under the 45 degree line. It's the curve with the shape of f(x)=-x^2 is "tangent" to the 45 degree line. By "tangent" I should have meant that those points around the "tangent" point are actually above the 45 degree line, very slightly though. Therefore, visually speaking, most of the data (~98%) are below the 45 degree line...

These are the residual QQ plot coming out from "plot(lmModel)"...

Using qqnorm(lmModel$res); qqline(lmModel$res) I got exactly the same curves and lines.

My questions are:

1. If my end-goal is to use yhat to do prediction onto a wide data-set, does the non-normality of data matter?

2. For Model Comparison, the approaches pointed out by Macro probably won't work; any other alternative approaches that don't assume Gaussian distribution?

3. What shall I do to fix the non-normality problem? (data-size: 10 variables, 1700 observations).

Thank you!

-
Of general interest are two threads: stats.stackexchange.com/questions/2492/… and stats.stackexchange.com/questions/29731/… which may be useful - these are mostly related to (1). There are various threads on transformation so you may want to do a search to find something specific, which would mostly address (3). –  Macro Jul 5 '12 at 20:00
Thank you Macro. However the two threads you posted above don't address the concern about prediction. Am I missing anything there? Thank you again! –  Luna Jul 5 '12 at 22:04

1. Because you are using least squares the nonnormality effects the regression coefficients and can hurt prediction. You may want to try a robust regression method.

2. Criteria like AIC and BIC look at closeness of the fit penalized by the number of parameters used. I do not think that the normality is important in choosing between models using this type of criteria. But keep in mind that if all these models have nonnormal residuals the fact that they all use least squares may mean that they all could be improved using a more robust fitting technique.

3. If you apply robust regression you do not have to "fix the nonnormality problem". Finding suitable transformations for the covariates in the model might be a way to "fix the nonnormality problem." But appropriate transformations may not be apparent.

-
Thank you! Why does the non-normality hurt the prediction? Does it ever "help" the prediction? How does a "robust regression" method help in this case? Thank you again! –  Luna Jul 5 '12 at 22:06
@Luna The regression coefficients based on least squares are sensitve to outliers. So the slope of the regression is forced to fit outlying observations. This is because least squares minimizes the sum of squared errors. Robust regression use different fitting criteria that don't peanlize as much for large individual errors. For example one robust method uses the sum of the absolute value of the errors. This will not as large . For example a term with an absolute error of 2 would have a squared error of 4. –  Michael Chernick Jul 5 '12 at 22:18
I can not conceive of a situation where a least squares fit to nonnormal data will improve prediction. –  Michael Chernick Jul 5 '12 at 22:20
Thank you Mike. But I don't have outliers. It is just I am having highly non-normal data and I was wondering what's an optimal solution under non-normal data and how does it impact my yhat? Thanks again! –  Luna Jul 5 '12 at 22:22
Nonnormality can occur because of heavy skewness or heavy kurtosis. If you have heavy kurtosis there are probably some outliers in the data. They don't necessarily show up as large residuals. –  Michael Chernick Jul 5 '12 at 22:33
show 3 more comments

The residual plot that you describe sounds like a right skewed distribution. One possibility is to fit a regression model that assumes a right skewed distribution rather than a normal distribution. The glm function can be used to fit a gamma distribution (which is right skewed).

Another approach is to transform the data, a log tranform on the y-variable or other Box-Cox transforms can help with skewness.

The biggest problem with skewed data and regression is that the usual tests are based on normality, so you can fit the regression model using regular least squares or robust methods, then instead of the normal based tests use permutation or bootstrap tests that do not depend on normality (but make sure you understand what assumptions you are making).

For any of these make sure that they make sense with the science and the questions that you are asking.

-