Impact of regression normality-assumption on model comparison & prediction? This question is a continuation of the discussion here:
How to test the statistical significance for categorical variable in linear regression?
Following Macro's suggestion, I started a new thread.
The new question is not limited to only study the inclusion/exclusion of categorical variable. 
It's about general model comparison and prediction. 
I found that my data is highly non-normal. The QQ plot is as follows: the curve is all below the straight 45 degree line. The curve is tangent to that straight line. And the curve looks like the curve of f(x)=-x^2 ( shape-wise). 
It's not the entire set of points that's under the 45 degree line. It's the curve with the shape of f(x)=-x^2 is "tangent" to the 45 degree line. By "tangent" I should have meant that those points around the "tangent" point are actually above the 45 degree line, very slightly though. Therefore, visually speaking, most of the data (~98%) are below the 45 degree line... 
These are the residual QQ plot coming out from "plot(lmModel)"... 
Using qqnorm(lmModel$res); qqline(lmModel$res) I got exactly the same curves and lines. 
My questions are:


*

*If my end-goal is to use yhat to do prediction onto a wide data-set, does the non-normality of data matter?

*For Model Comparison, the approaches pointed out by Macro probably won't work; any other alternative approaches that don't assume Gaussian distribution?

*What shall I do to fix the non-normality problem? (data-size: 10 variables, 1700 observations). 
Thank you!
 A: *

*Because you are using least squares the nonnormality effects the regression coefficients and can hurt prediction.  You may want to try a robust regression method.

*Criteria like AIC and BIC look at closeness of the fit penalized by the number of parameters used. I do not think that the normality is important in choosing between models using this type of criteria. But keep in mind that if all these models have nonnormal residuals the fact that they all use least squares may mean that they all could be improved using a more robust fitting technique.

*If you apply robust regression you do not have to "fix the nonnormality problem".  Finding suitable transformations for the covariates in the model might be a way to "fix the nonnormality problem." But appropriate transformations may not be apparent.
A: The residual plot that you describe sounds like a right skewed distribution.  One possibility is to fit a regression model that assumes a right skewed distribution rather than a normal distribution.  The glm function can be used to fit a gamma distribution (which is right skewed).
Another approach is to transform the data, a log tranform on the y-variable or other Box-Cox transforms can help with skewness.
The biggest problem with skewed data and regression is that the usual tests are based on normality, so you can fit the regression model using regular least squares or robust methods, then instead of the normal based tests use permutation or bootstrap tests that do not depend on normality (but make sure you understand what assumptions you are  making).
For any of these make sure that they make sense with the science and the questions that you are asking.
