The literature suggests that we need to have dataset that meets the condition of homoscedasticity. However, it seems that such a condition is not proper.
3 Answers
$\begingroup$
$\endgroup$
2
The classical linear regression model is $$Y_i = \beta_0 + \beta_1 X_{i,1} + \cdots + \beta_m X_{i,m} + \varepsilon_i$$ where the $\varepsilon_i$ are independent $\mathrm{normal}(0,\sigma^2)$ variables. The homoscedasticity requirement is a part of the condition on the $\varepsilon_i$, that they all have the same variance $\sigma^2$. This translates to the $Y_i$ having equal variance $\sigma^2$ given the $X_i$, i.e., the $Y_i$ at one value of $X$ has the same variance as $Y_i$ at another value of $X$. This does not imply that the sample variances of the dependent variable and independent variables are equal.
So no, homoscedasticity is a condition only on the dependent variable.
-
$\begingroup$ Residuals cannot "all have the same variance" for example, a not-randomly selected grouping of the 10 residuals closest in value will have lower variance than the 10 residuals most spread out across the range of residuals. Homoscedasticity is specifically the point that variance in Y is constant across values of X. $\endgroup$– AlexisApr 28, 2014 at 14:17
-
4$\begingroup$ The residuals cannot all have the same variance, yes, but the $\varepsilon_i$ are the errors, which are assumed to be iid. The residuals are the observed values minus the fitted value, which is different from the errors, the observed values minus the unobservable true means of the $Y$ at the given value of $X$. $\endgroup$ Apr 28, 2014 at 14:39
$\begingroup$
$\endgroup$
4
I don't know if this should go in a comment instead, but I wanted to point the following out:
Under the Gauss-Markov assumptions, the OLS estimator is the best linear unbiased estimator. Here, "best" means minimizing variance. However, if we deviate from the assumption of homoskedasticity, i.e. a constant variance on the errors, the following happens
- OLS is still unbiased.
- It is no longer the minimum-variance estimator, i.e. it is inefficient.
- The estimate of the variance of the coefficients - $\hat{V(\hat{\beta})}$ becomes biased, which means that our standard inferential techniques - the $t$ and $F$-tests - becomes invalid.
-
$\begingroup$ But doesn't Anscombe's famous example illustrates precisely why $\beta$ itself is biased by heteroscedasticity? In his article the entire slope is defined by high-leverage (outlier on $x$ plus low variability on $y$); there is clearly zero slope for the vast majority of the data. $\endgroup$– AlexisApr 28, 2014 at 17:20
-
4$\begingroup$ @Alexis, that an estimate is wrong in a given sample does not imply that the estimator is biased. Consider a case, where we have a single outlier with much high variance: sometimes, this will lead to $\hat{\beta}$ being higher than $\beta$, sometimes lower. However, on average, it shouldn't make it differ (thus the unbiasedness). $\endgroup$– abaumannApr 28, 2014 at 18:01
-
1$\begingroup$ That really clarifies... bias is one of those words with so many definitions. $\endgroup$– AlexisApr 28, 2014 at 18:12
-
4$\begingroup$ @Alexis: "bias" is a characteristic of an estimator, not an estimate. Wiki has a good summary at en.wikipedia.org/wiki/Bias_of_an_estimator $\endgroup$– abaumannApr 29, 2014 at 11:46
$\begingroup$
$\endgroup$
You misunderstand homoscedasticity. Y and X can certainly have different variances in ordinary least squares (OLS) regression. The homoscedasticity assumption simply means that variance in Y is constant across different values of X.
See Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1):17–21. For a good run down on this and three other OLS regression assumptions (he doesn't go into visuals on the independently and identically distributed assumption... no random walks, but it's a great introductory read).
