2
$\begingroup$

I'm working on a Kaggle multiple regression tutorial competition and inspecting plots of my residuals. I followed a suggestion and log transformed several independent variables and the dependent variable. These are the plots I got after fitting a Ridge regression model (sample size is 1500):

enter image description here

enter image description here

I'm trying to determine how to interpret these plots, and to better understand which is useful for which purposes. I believe that the first two plots illustrate that the regression assumptions of linearity, additivity, and homoscedasticity are not violated, although at lower prices my model tends to underestimate. I think that the pattern in the QQ-plot shows a distribution more heavily tailed than normal, so the variance is higher than expected for a normal distribution, and so the normality of error assumption is violated. Is this a correct assessment? And if my goal is prediction rather than inference, should I be concerned with this QQ-plot? I also did not divide the residuals by their standard deviation because I could not find enough information about when this step is necessary.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ To me the summary line on the first graph is tilted. Is that an illusion? Should be residual = 0. $\endgroup$
    – Nick Cox
    Sep 20, 2017 at 21:20
  • $\begingroup$ It does seem slightly tilted. Is this a concern I should address in any particular way to build a more accurate model? $\endgroup$
    – Austin
    Sep 20, 2017 at 21:24
  • $\begingroup$ Tell us how it is produced! (Someone familiar with ridge regression may be able to explain that it need not be flat -- I have never used it.) $\endgroup$
    – Nick Cox
    Sep 20, 2017 at 21:27
  • 1
    $\begingroup$ Thanks, but syntax in unspecified software doesn't explain to most people how the line was calculated. Also, although it is clear from plain regression that residuals have mean zero, I am asking openly whether that is also true for the ridge flavour. $\endgroup$
    – Nick Cox
    Sep 20, 2017 at 21:35
  • 2
    $\begingroup$ @NIck With shrinkage (using ridge regression) you would expect the residual plot would not necessarily be quite flat (though I'd have thought it would go the other way) $\endgroup$
    – Glen_b
    Sep 20, 2017 at 22:47

1 Answer 1

Reset to default
4
$\begingroup$

Yes. To me, your top plots look pretty good. Your qq-plot shows clear non-normality / fat tails. The histogram / density plot looks pretty symmetrical, it's just that you have 'too many' residuals that are too far from the predicted line. This means the kurtosis is too large, not that the residual variance is. The variance is a parameter of a normal distribution to be fitted, so it cannot be too large.

The effect of non-normality is somewhat complex. When you want to make inferences, it can mean your p-values are wrong, but you appear to have a good amount of data, so the central limit theorem may kick in enough that it doesn't matter. If you only care about predicted means, it shouldn't have much impact. But I suspect it is likely you will want to know something about the prediction intervals as well as the means. Standard prediction intervals are based much more closely on the idea that the conditional distribution is normal than the confidence intervals. For example, the central limit theorem cannot save your prediction intervals no matter how much data you have. You might see if there is a suitable fat-tailed distribution (e.g., a low df t-distribution) that is a good fit for your residuals that you could use for forming prediction intervals.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks very much for your answer. I edited my question to include sample size (1500). My primary goal is to submit the lowest RMSE value I can with a predictive model in order to rank well in a Kaggle competition. I haven't really learned about prediction intervals, so I'm not quite sure how/if there's a way to use them in order to assess and build a more accurate model. I do know about confidence intervals which appear to be fairly related, and don't immediately see a value for my purpose, but I'm also quite new to this. $\endgroup$
    – Austin
    Sep 20, 2017 at 21:23
  • $\begingroup$ +1 The central limit theorem may give you accurate type I error rate, but it won't help with power; on the other hand I doubt power is a prime consideration here (it would be prediction for this rather than testing hypotheses), but I think it would also affect size of confidence intervals (making them wider than needed? I am a bit overtired so I may have flipped that). Even if we supposed this was (say) a symmetric mixture I don't think that will change much if you're trying to minimize MSE. Out of sample selection and tuning on your criterion should still get you to about the right place. $\endgroup$
    – Glen_b
    Sep 20, 2017 at 22:56
  • 1
    $\begingroup$ @Jake, maybe my answer here: Linear regression prediction interval might help. $\endgroup$
    – gung - Reinstate Monica
    Sep 21, 2017 at 0:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.