What does having "constant variance" in the error term mean? As I see it, we have a data with one dependent variable and one independent variable. Constant variance is one of the assumptions of linear regression. I am wondering what homoscedasticity means. Since even if I have 500 rows, I would have a single variance value which is obviously constant. With what variable should I compare the variance?


It means that when you plot the individual error against the predicted value, the variance of the error predicted value should be constant. See the red arrows in the picture below, the length of the red lines (a proxy of its variance) are the same.

enter image description here

| cite | improve this answer | |
  • 1
    $\begingroup$ Ok Understood.!! But Since it is an assumption don't we need to validate the assumption before running the model. And why do we need this assumption $\endgroup$ – Mukul Mar 13 '13 at 13:13
  • 2
    $\begingroup$ Some assumptions can only be tested after the model is run. Calculating a model is just math and not the same as interpreting a model. $\endgroup$ – John Mar 13 '13 at 13:20
  • 6
    $\begingroup$ Range does not equal variance Penguin Knight so you might want to update your wording here. $\endgroup$ – John Mar 13 '13 at 13:21
  • 4
    $\begingroup$ If your variance assumption is wrong, then it will usually mean that the standard errors are wrong and any hypothesis testing could draw the wrong conclusions. (A different John) $\endgroup$ – John Mar 13 '13 at 13:43
  • 4
    $\begingroup$ I differ slightly. I wouldn't say that heteroscedasticity necessarily means the standard errors of your betas are wrong, but rather that the OLS estimator is no longer the most efficient unbiased estimator. That is, you could get more power / precision if either you had constant variance (perhaps due to a transformation of Y), or if you accurately took the non-constancy into account (perhaps via the generalized least squares estimator). $\endgroup$ – gung - Reinstate Monica Mar 13 '13 at 15:35

This is a place where I've found looking at some formulas helps, even for people with some math anxiety (I'm not suggesting that you do, necessarily). The simple linear regression model is this:
$$ Y=\beta_0+\beta_1X+\varepsilon \\ \text{where } \varepsilon\sim\mathcal N(0, \sigma^2_\varepsilon) $$ What's important to note here is that this model explicitly states once you've estimated the meaningful information in the data (that's the "$\beta_0+\beta_1X$") there is nothing left over but white noise. Moreover, the errors are distributed as a Normal with a variance of $\sigma^2_\varepsilon$.

It's important to realize that $\sigma^2_\varepsilon$ is not a variable (although in junior high school level algebra, we would call it that). It doesn't vary. $X$ varies. $Y$ varies. The error term, $\varepsilon$, varies randomly; that is, it is a random variable. However, the parameters ($\beta_0,~\beta_1,~\sigma^2_\varepsilon)$ are placeholders for values we don't know--they don't vary. Instead, they are unknown constants. The upshot of this fact for this discussion is that no matter what $X$ is (i.e., what value is plugged in there), $\sigma^2_\varepsilon$ remains the same. In other words, the variance of the errors / residuals is constant. For the sake of contrast (and perhaps greater clarity), consider this model:
$$ Y=\beta_0+\beta_1X+\varepsilon \\ \text{where } \varepsilon\sim\mathcal N(0, f(X)) \\ ~ \\ \text{where } f(X)=\exp(\gamma_0+\gamma_1 X) \\ \text{and }\gamma_1\ne 0 $$ In this case, we plug in a value for $X$ (starting on the third line), pass it through the function $f(X)$ and get the error variance that obtains at that exact value of $X$. Then we move through the rest of the equation as usual.

The above discussion should help with understanding the nature of the assumption; the question also asks about how to assess the it. There are basically two approaches: formal hypothesis tests and examining plots. Tests for heteroscedasticity can be used if you have experimental-ish data (i.e., that only occur at fixed values of $X$) or an ANOVA. I discuss some such tests here: Why Levene test of equality of variances rather than F-ratio. However, I tend to think looking at plots is best. @Penquin_Knight has done a good job of showing what constant variance looks like by plotting the residuals of a model where homoscedasticity obtains against the fitted values. Heteroscedasticity can also possibly be detected in a plot of the raw data, or in a scale-location (also called spread-level) plot. R conveniently plots the latter for you with a call to plot.lm(model, which=2); it is the square root of the absolute values of the residuals against the fitted values, with a lowess curve helpfully overlaid. You want the lowess fit to be flat, not sloped.

Consider the plots below, which compare how homoscedastic vs. heteroscedastic data might look in these three different types of figures. Note the funnel shape for the upper two heteroscedastic plots, and the upward sloping lowess line in the last one.

enter image description here

For completeness, here is the code that I used to generate these data:


N  = 500
b0 = 3
b1 = 0.4

s2 = 5
g1 = 1.5
g2 = 0.015

x        = runif(N, min=0, max=100)
y_homo   = b0 + b1*x + rnorm(N, mean=0, sd=sqrt(s2            ))
y_hetero = b0 + b1*x + rnorm(N, mean=0, sd=sqrt(exp(g1 + g2*x)))

mod.homo   = lm(y_homo~x)
mod.hetero = lm(y_hetero~x)
| cite | improve this answer | |
  • 1
    $\begingroup$ thanks it is very helpful . Can you also explain why do we need this assumption in a layman language $\endgroup$ – Mukul Mar 14 '13 at 8:36
  • 5
    $\begingroup$ You're welcome, @Mukul. The assumption of homoscedasticity (constant variance) is required to make the OLS estimator (ie, the default procedure software uses to estimate betas) the estimation procedure that will produce sampling distributions of betas that have the narrowest standard errors of all the estimation procedures that yield sampling distributions which are centered on the true value. IE, it is necessary for the OLS estimator to be the minimum variance unbiased estimator. $\endgroup$ – gung - Reinstate Monica Mar 14 '13 at 17:24
  • 5
    $\begingroup$ If your response variable is binary, it will be distributed as a binomial. IE, many parts of the linear regression model described above are inappropriate. 1 of those issues is that, since the variance of a binomial is a function of the mean (mean: $p$, variance: $(p(1-p))/n)$ ), the assumption of homoscedasticity is violated. To understand these things better, it may help to read my answer here: difference-between-logit-and-probit-models, although it was written in a different context. $\endgroup$ – gung - Reinstate Monica Mar 14 '13 at 17:30
  • 3
    $\begingroup$ @gung in your comment you put italics on all the words in the phrase minimum variance unbiased estimator. I understand that with heteroscedasticity the estimator will become less efficient (more variance), but will it become biased too? $\endgroup$ – user1205901 - Reinstate Monica May 24 '15 at 2:47
  • 5
    $\begingroup$ @user1205901, it remains unbiased. $\endgroup$ – gung - Reinstate Monica May 24 '15 at 2:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.