Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

What does having "constant variance" in the error term means. As I see it , we have a data with variable and 1 independent variable. This is one assumption of linear regression. I am wondering what this homoscedasticity means. Since if I am having 500 rows I would have single variance value which is obviously constant. With what variable should I compare the variance?

share|improve this question

2 Answers 2

up vote 15 down vote accepted

It means that when you plot the individual error against the predicted value, the variance of the error predicted value should be constant. See the red arrows in the picture below, the length of the red lines (a proxy of its variance) are the same.

enter image description here

share|improve this answer
    
Ok Understood.!! But Since it is an assumption don't we need to validate the assumption before running the model. And why do we need this assumption –  Mukul Mar 13 '13 at 13:13
1  
Some assumptions can only be tested after the model is run. Calculating a model is just math and not the same as interpreting a model. –  John Mar 13 '13 at 13:20
5  
Range does not equal variance Penguin Knight so you might want to update your wording here. –  John Mar 13 '13 at 13:21
2  
If your variance assumption is wrong, then it will usually mean that the standard errors are wrong and any hypothesis testing could draw the wrong conclusions. (A different John) –  John Mar 13 '13 at 13:43
1  
I differ slightly. I wouldn't say that heteroscedasticity necessarily means the standard errors of your betas are wrong, but rather that the OLS estimator is no longer the most efficient unbiased estimator. That is, you could get more power / precision if either you had constant variance (perhaps due to a transformation of Y), or if you accurately took the non-constancy into account (perhaps via the generalized least squares estimator). –  gung Mar 13 '13 at 15:35

This is a place where I've found looking at some formulas helps, even for people with some math anxiety (I'm not suggesting that you do, necessarily). The simple linear regression model is this:
$$ Y=\beta_0+\beta_1X+\varepsilon \\ \text{where } \varepsilon\sim\mathcal N(0, \sigma^2_\varepsilon) $$ What's important to note here is that this model explicitly states once you've estimated the meaningful information in the data (that's the "$\beta_0+\beta_1X$") there is nothing left over but white noise. Moreover, the errors are distributed as a Normal with a variance of $\sigma^2_\varepsilon$.

It's important to realize that $\sigma^2_\varepsilon$ is not a variable (although in junior high school level algebra, we would call it that). It doesn't vary. $X$ varies. $Y$ varies. The error term, $\varepsilon$, varies randomly; that is, it is a random variable. However, the parameters ($\beta_0,~\beta_1,~\sigma^2_\varepsilon)$ are placeholders for values we don't know--they don't vary. Instead, they are unknown constants. The upshot of this fact for this discussion is that no matter what $X$ is (i.e., what value is plugged in there), $\sigma^2_\varepsilon$ remains the same. In other words, the variance of the errors / residuals is constant. For the sake of contrast (and perhaps greater clarity), consider this model:
$$ Y=\beta_0+\beta_1X+\varepsilon \\ \text{where } \varepsilon\sim\mathcal N(0, f(X)) \\ ~ \\ \text{where } f(X)=\exp(\gamma_0+\gamma_1 X) \\ \text{and }\gamma_1\ne 0 $$ In this case, we plug in a value for $X$ (starting on the third line), pass it through the function $f(X)$ and get the error variance that obtains at that exact value of $X$. Then we move through the rest of the equation as usual.


The above discussion should help with understanding the nature of the assumption; the question also asks about how to assess the it. There are basically two approaches: formal hypothesis tests and examining plots. Tests for heteroscedasticity can be used if you have experimental-ish data (i.e., that only occur at fixed values of $X$) or an ANOVA. I discuss some such tests here: Why Levene test of equality of variances rather than F-ratio. However, I tend to think looking at plots is best. @Penquin_Knight has done a good job of showing what constant variance looks like by plotting the residuals of a model where homoscedasticity obtains against the fitted values. Heteroscedasticity can also possibly be detected in a plot of the raw data, or in a scale-location (also called spread-level) plot. R conveniently plots the latter for you with a call to plot.lm(model, which=2); it is the square root of the absolute values of the residuals against the fitted values, with a lowess curve helpfully overlaid. You want the lowess fit to be flat, not sloped.

Consider the plots below, which compare how homoscedastic vs. heteroscedastic data might look in these three different types of figures. Note the funnel shape for the upper two heteroscedastic plots, and the upward sloping lowess line in the last one.

enter image description here

For completeness, here is the code that I used to generate these data:

set.seed(5)

N  = 500
b0 = 3
b1 = 0.4

s2 = 5
g1 = 1.5
g2 = 0.015

x        = runif(N, min=0, max=100)
y_homo   = b0 + b1*x + rnorm(N, mean=0, sd=sqrt(s2            ))
y_hetero = b0 + b1*x + rnorm(N, mean=0, sd=sqrt(exp(g1 + g2*x)))

mod.homo   = lm(y_homo~x)
mod.hetero = lm(y_hetero~x)
share|improve this answer
    
thanks it is very helpful . Can you also explain why do we need this assumption in a layman language –  Mukul Mar 14 '13 at 8:36
    
Lemme rephrase my question : If we apply linear regression on a data which has a BINARY(0,1) dependant variable, the very important assumption of "constant variance" of the dependant variable across independant variables is violated. Can you explain how ? –  Mukul Mar 14 '13 at 13:16
1  
You're welcome, @Mukul. The assumption of homoscedasticity (constant variance) is required to make the OLS estimator (ie, the default procedure software uses to estimate betas) the estimation procedure that will produce sampling distributions of betas that have the narrowest standard errors of all the estimation procedures that yield sampling distributions which are centered on the true value. IE, it is necessary for the OLS estimator to be the minimum variance unbiased estimator. –  gung Mar 14 '13 at 17:24
1  
If your response variable is binary, it will be distributed as a binomial. IE, many parts of the linear regression model described above are inappropriate. 1 of those issues is that, since the variance of a binomial is a function of the mean (mean: $p$, variance: $(p(1-p))/n)$ ), the assumption of homoscedasticity is violated. To understand these things better, it may help to read my answer here: difference-between-logit-and-probit-models, although it was written in a different context. –  gung Mar 14 '13 at 17:30
    
Appreciate your reply .It is of great help :) –  Mukul Mar 14 '13 at 19:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.