11
$\begingroup$

I am performing the multiple linear regression below in R to predict returns on fund managed.

reg <- lm(formula=RET~GRI+SAT+MBA+AGE+TEN, data=rawdata)

Here only GRI & MBA are binary/dichotomous predictors; the remaining predictors are continuous.

I am using this code to generate residual plots for the binary variables.

plot(rawdata$GRI, reg$residuals)
abline(lm(reg$residuals~rawdata$GRI, data=rawdata), col="red") # regression line (y~x) 

plot(rawdata$MBA, reg$residuals)
abline(lm(reg$residuals~rawdata$MBA, data=rawdata), col="red") # regression line (y~x) 

My Question: I know how to inspect residual plots for continuous predictors but how do you test assumptions of linear regression such as homoscedasticity when an independent variable is binary?

Residual Plots:

Residual Plot for GR1 Residual Plot for MBA

$\endgroup$
8
$\begingroup$

@NickCox has done a good job talking about displays of residuals when you have two groups. Let me address some of the explicit questions and implicit assumptions that lie behind this thread.

The question asks, "how do you test assumptions of linear regression such as homoscedasticity when an independent variable is binary?" You have a multiple regression model. A (multiple) regression model assumes there is only one error term, which is constant everywhere. It isn't terribly meaningful (and you don't have) to check for heteroscedasticity for each predictor individually. This is why, when we have a multiple regression model, we diagnose heteroscedasticity from plots of the residuals vs. the predicted values. Probably the most helpful plot for this purpose is a scale-location plot (also called 'spread-level'), which is a plot of the square root of the absolute value of the residuals vs. the predicted values. To see examples, look at the bottom row of my answer here: What does having "constant variance" in a linear regression model mean?

Likewise, you don't have to check the residuals for each predictor for normality. (I honestly don't even know how that would work.)

What you can do with plots of residuals against individual predictors is check to see if the functional form is properly specified. For example, if the residuals form a parabola, there is some curvature in the data that you have missed. To see an example, look at the second plot in @Glen_b's answer here: Checking model quality in linear regression. However, these issues don't apply with a binary predictor.

For what it's worth, if you only have categorical predictors, you can test for heteroscedasticity. You just use Levene's test. I discuss it here: Why Levene's test of equality of variances rather than F ratio? In R you use ?leveneTest from the car package.


Edit: To better illustrate the point that looking at a plot of the residuals vs. an individual predictor variable does not help when you have a multiple regression model, consider this example:

set.seed(8603)                       # this makes the example exactly reproducible
x1 = sort(runif(48, min=0, max=50))  # here is the (continuous) x1 variable
x2 = rep(c(1,0,0,1), each=12)        # here is the (dichotomous) x2 variable
y  = 5 + 1*x1 + 2*x2 + rnorm(48)     # the true data generating process, there is 
                                     #   no heteroscedasticity

mod = lm(y~x1+x2)                    # this fits the model

You can see from the data generating process that there is no heteroscedasticity. Let's examine the relevant plots of the model to see if they imply problematic heteroscedasticity:

enter image description here

Nope, nothing to worry about. However, let's look at the plot of the residuals vs. the individual binary predictor variable to see if it looks like there is heteroscedasticity there:

enter image description here

Uh oh, it does look like there may be a problem. We know from the data generating process that there isn't any heteroscedasticity, and the primary plots for exploring this didn't show any either, so what is happening here? Maybe these plots will help:

enter image description here

x1 and x2 are not independent of each other. Moreover, the observations where x2 = 1 are at the extremes. They have more leverage, so their residuals are naturally smaller. Nonetheless, there is no heteroscedasticity.

The take home message: Your best bet is to only diagnose heteroscedasticity from the appropriate plots (the residuals vs. fitted plot, and the spread-level plot).

$\endgroup$
  • $\begingroup$ Thanks! For the same regression I was doing I found that Residual Vs Y is homoscedastic but when I checked Residual Vs tenure(independent) it was a funnel shape. So i need to do some transformation to correct this right? Then in this context just wanted to understand why you mentioned that checking residual Vs independent variable is not necessary? $\endgroup$ – GeorgeOfTheRF Oct 14 '14 at 4:23
  • $\begingroup$ @mrcet007, no you don't need a transformation. If the res vs fitted shows no heteroscedasticity, you are OK. Perhaps an illustration will help you. I've edited my answer to add a demonstration. $\endgroup$ – gung - Reinstate Monica Oct 15 '14 at 2:05
  • $\begingroup$ Can you check this link people.duke.edu/~rnau/testing.htm . It says check residual Vs independent variable also. Just sharing for discussions sake. Can you comment on this? What i was thinking was we need to always check both residual Vs predicted as well as residual vs independent. homoscedasticity (constant variance) of the errors (a) versus time (in the case of time series data) (b) versus the predictions (c) versus any independent variable $\endgroup$ – GeorgeOfTheRF Oct 17 '14 at 13:45
  • $\begingroup$ My comment is that I have provided you with both a reason why you look at residual vs predicted graphs to check for heteroscedasticity & showed you an example of how looking at residual vs IV graphs can lead you astray. I don't know what else there is to say. $\endgroup$ – gung - Reinstate Monica Oct 17 '14 at 23:57
6
$\begingroup$

It's true that conventional residual plots are harder work in this case: it can be (much) more difficult to see whether the distributions are about the same. But there are easy alternatives here. You are just comparing two distributions, and there are many good ways to do that. Some possibilities are side-by-side or superimposed quantile plots, histograms or box plots. My own prejudice is that box plots unadorned are often over-used here: they usually will suppress the detail we should want to look at, even if we can often dismiss it as unimportant. But you can eat your cake and have it.

You use R, but nothing statistical in your question is R-specific. Here I used Stata for a regression on a single binary predictor and then fired up quantile box plots comparing the residuals for the two levels of the predictor. The practical conclusion in this example is that the distributions are about the same.

enter image description here

More detail if the plot looks cryptic: For each distribution, we have a quantile plot, i.e. the ordered values are plotted versus their (fractional) rank. A box showing median and quartiles is superimposed. Hence each box is defined vertically in the usual way and horizontally because it is bounded by lines for fractional ranks $1/4$ and $3/4$.

Note: See also How to present box plot with an extreme outlier? including @Glen_b's example of similar plots using R. Such plots should be easy in any decent software; if not, your software is not decent.

$\endgroup$
  • $\begingroup$ +1 Beautiful. Do you feel that there is a role for hypothesis testing on the residuals here as well? $\endgroup$ – Alexis Oct 6 '14 at 23:43
  • $\begingroup$ @gung I edited your edit. The original was evidently not clear enough if you misunderstood it. $\endgroup$ – Nick Cox Oct 6 '14 at 23:46
  • 2
    $\begingroup$ @Alexis Thanks! I am happy with the idea that a hypothesis of equal scatter is supported informally by the graph in this case. I am not of the school of thought that every small step in an analysis needs to be sanctified by a P-value. Unfortunately, it is never easy to be sure that you jump the right way, but I would in practice entertain other models too if I was in doubt. Here the example is just concocted for the question and not part of a serious analysis. $\endgroup$ – Nick Cox Oct 6 '14 at 23:48
  • $\begingroup$ My apologies, Nick. I misunderstood the point of that phrase. I thought it was a typo. It is clearer now. $\endgroup$ – gung - Reinstate Monica Oct 7 '14 at 1:20
  • 1
    $\begingroup$ @whuber That's fine by me. Some people find them confusing, or so I am told. $\endgroup$ – Nick Cox Oct 7 '14 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.