# Wrong predictions in linear regression

My predictions are far off the actual values, and I really do not understand what I am doing wrong. I have 20 dummy independent variables, and a continuous dependent variable. The dataset is small, 1000 points approx. The best result I got so far is using generalized linear regression, Poisson link function, and include all interactions terms. Only after introducing interactions terms, I actually find that some (less than 50%) of my prediction actually somewhat follow the true value. What else could I be missing to explain the remaining data? Could it be that simply my independent variables are not good predictors for the remaining part of the data?

Below I show you the predicted (y) vs observed (x) dependent variable, the residuals (y) vs actual dependent variable (x), and the residual (y) vs the predicted dependent variable (x). • (1) Look at the model you obtain when you remove that outlier (with an outcome of 1.0). (2) Consider a different model designed to handle the evident discrete nature of the outcome (it focuses on particular values near 0, 0.19, and 0.38).
– whuber
Mar 10 '17 at 15:54
• As well as @whuber suggestions you need to consider that you have some points in sparsely populated areas of the space of the X's. Your residual plot does not look that bad to me, modulo the point with large negative residual. Mar 10 '17 at 16:03
• @mdewey, why do you say it does not look bad? To me the simplest plot to interpret is the second, that shows that if the "true" relation was y= x, I am really predicting something like y=0.5. That is a large bias. Mar 10 '17 at 16:11
• Sorry, I meant the third plot, the conventional one for examining the assumptions, not the second one. Mar 10 '17 at 16:13
• I guess I do not understand well the third plot. Sure enough, the model as is is clearly wrong, unless one is willing to calibrate it using the middle plot. BTW, I edited the question to remove the outliers. I can only see the same mess, but closer :( Mar 10 '17 at 16:15

The third plot is the most useful. Your data is showing heteroskedasticity, that is, the variance of the error term is not constant. It is clear by the third plot. Your error should be randomly distributed around zero. The others plots are not very useful to me at this point. Plot $\sqrt{\hat{e}}$ (e is the error) vs $\hat{y}$ too and check it. I suggest to use Generalized LS estimator, or try a transformation of $y$, either log or Cox transformation.