# Multiple Regression - Residual Analysis

I am doing a multiple regression analysis regressing GPA against several 0/1 indicator variables (representing course completions). My fitted vs. residual plot is biased and looks awful. See below. Is there some way to make this better besides adding more predictors?

Also, I am not trying to do any prediction. I only want to know if the predictors are significant and what their effect size is (strength of relationship). So, how would the biased residuals affect this?

Any help is appreciated. Thanks!!

• The evident limits in the upper plot are due to the constraint that GPA lie between $0$ and $4$ (or $60$ and $100$ or whatever the limits are). That phenomenon will be difficult to remove, so concentrate on other issues first. One interesting one may be interactions. How many binary variables do you have? If it is a small number compared to the logarithm of the number of cases you have, consider fitting a model with all possible interactions (the "saturated" model), if only to establish (for reference) what a good-looking residual plot should look like.
– whuber
Oct 8 '14 at 22:11
• @whuber, would it make sense to multiply GPA by some value (say, 100) and use logistic regression (allowing for 0 to 400 "hits" or so)?
– jona
Oct 9 '14 at 7:32
• Hey whuber. Thanks for the suggestion. I have 13 binary variables. I ran a model with all first order interactions and the fits to residual plot looks essentially the same. Any other suggestions? Can you tell me how exactly this affects my analysis? Thank you! Oct 9 '14 at 14:32
• @jona Yes and no. Some people have advocated applying a logistic GLM even to non-count data, essentially as an ad hoc (but sometimes very effective way) to handle this form of heteroscedasticity. Because a GPA isn't a multiple of an actual count there's no need, or reason, to rescale it a priori, but nevertheless your suggestion is intriguing: scaling the values down would allow for more variability; scaling them up for less. Such a scale factor could be incorporated as a parameter in the model to cope with the residuals. And that, more or less, describes Beta regression.
– whuber
Oct 9 '14 at 16:25
• @jona Try rseek.org/…. Googling works just fine for me, too: google.com/…
– whuber
Oct 10 '14 at 14:34

## 1 Answer

It looks like you're adding the binary indicator variables and using the sum to predict GPA, if I'm interpreting your first graphic correctly. While this may work in some situations, it clearly doesn't seem to be effective in this case.

Instead, try something like:

model <- lm(data = DATA, y ~ binary1 + binary2 + binary3 + binaryN)


and then use

anova(model)


to look at significance and effect size of each course.

• Hey, thanks for the response. I forgot to mention I used a box-cox transformation. However, the plot looks the same with or without the transformation...only the fitted values change. Oct 8 '14 at 21:41
• So, basically I already did as you suggested. I didn't use a sum. I just used a transformed response. Oct 8 '14 at 21:50