# Understanding the causes and implication of heteroskedasticity

I’m trying to heteroskedasticity and how, even if we don’t have MLR 5 assumption (heteroskedasticity), we can still have unbiased estimates.

I was thinking: a very intuitive cause of a growing variance would be that we have an omitted variable bias. In this case, wouldn’t MLR 4 be violated as well, as thus the above statement is invalid?

(MLR 4 refers to conditional mean independence of X and error terms. I understand that this means that X and errors must be uncorrelated. Therefore an OVB would rule this out)

Am I just thinking of but one specific cause of heteroskedasticity, and thus the other causes can still preserve MLR 4? I understand another cause is autocorrelation, which is like inertia of an outcome with respect to an X variable, usually time (in time series).

• Details like MLR 4 assume that we studied from the same textbook or course notes and recall it literally. At a wild guess that is true for less than 1% of people reading your question. Better to give names -- which you do too. – Nick Cox Dec 1 '20 at 9:15

Here is a source out of the econometric literature to summarize some possible reasons for heteroskedasticity which include:

1. As people learn the error of their behavior, as in typing mistakes, there can be an improvement (reduction in errors) over time.

2. As income grows people have more choices on the disposition of their income.

3. As data collection techniques (as in automated data processing) improve, associated data collection errors are reduced.

4. Outliers can be a factor in heteroskedasticity.

5. A model specification error can contribute to heteroskedasticity as occurs in the emission of important variables.

6. Skewness in the distribution of a regressor variable could be a possible source for heteroskedasticity as occurs, for example, in the distribution of wealth and income, where there is a significant concentration in a few.

7. In modeling, possible errors related to heteroskedasticity could occur relating to inappropriate data transformations.

8. Or inappropriate functional form as in linear versus a log-linear model.

9. And finally, heteroskedasticity is more common in cross-sectional data than conventional time series analysis.

I hope this helps.

• +1 This is a good example of an answer that directly addresses the question, at an appropriate level of terminology and sophistication, and reflects good research. For completeness, would you mind citing your source explicitly? Over time, links rot, so a citation can help give credit where it's due and help future readers find a copy of your reference. – whuber Nov 30 '20 at 17:24
• Yes, I agree with your comment but alas, this is currently available (how long?) as a slide presentation 'Heteroskedasticity' by Amir Samimi. I thought highly of the quality of the presentation, hence the basis of my answer. – AJKOER Nov 30 '20 at 17:49
• That's OK: a citation would still be welcome. – whuber Nov 30 '20 at 17:50
• Have a look at stats.stackexchange.com/questions/434928/… which could be relevant for your answer! – kjetil b halvorsen Nov 30 '20 at 23:40

I find the "MLR X" format somewhat confusing, so I won't use it here. They basically refer to the conditions under which the Gauss-Markov Theorem applies, i.e. the conditions under which the OLS estimator is BLUE (best linear unbiased estimator). The most important ones are: 1) Regressors must be exogenous, i.e. $$E[u|X]=0$$, and 2) errors must be homoskedastic, i.e. $$Cov[u]=\sigma^2I$$ where $$I$$ is the identity matrix. I tried to break down your post into 2 main questions:

Question 1: How can we have unbiased estimators with heteroskedasticity? Answer: unbiasedness has nothing to do with the variance and is not the result of the Gauss-Markov theorem. For unbiasedness it exogeneity suffices, i.e. $$E[u|X]=0$$. Proof: $$E[\hat \beta_{OLS}|X]=E[(X^TX)^{-1}X^Ty|X] \\ = E[(X^TX)^{-1}X^T(X\beta + u)|X] \\ =(X^TX)^{-1}(X^TX)\beta+ \underbrace{E[u|X]}_{=0}=\beta$$

Notice how the variance of $$u$$ is not imporant here, i.e. it could be that $$Var[u_i]=\sigma^2_i$$ with a different $$\sigma^2_i$$ each time (i.e. heteroskedasticity). Further notice that exogeneity implies no correlation, i.e. $$u_i$$ can not be autocorrelated.

Question 2: Can the Gauss-Makrov Theorem still hold under heteroskedasticity? Answer: Under heteroskedasticity, the Gauss-Markov Theorem does not apply anymore, i.e. the OLS estimator is not BLUE anymore. It still a LUE, i.e. linear and unbiased, but not best anymore. Here "best" refers to the estimator with the lowest variance. The GLS (generalized least squares) estimator is BLUE now instead, if $$\Omega$$ is any variance-covariance matrix, and not just $$\Omega=\sigma^2 I$$ anymore: $$\beta_{GLS}=(X^T\Omega^{-1} X)^{-1}X^T \Omega^{-1}X$$. Proof is somewhat lengthy, but not hard. Essentially it boils down to proving that $$Var[\beta_{OLS}]-Var[\beta_{GLS}]\ge0$$ for any $$\Omega$$.

• Regarding your answer to Question 2, does this imply that the SE of beta_OLS under heteroskedasticity will be underestimated and so the confidence interval will have less than 100% coverage? Or does it simply mean that the SE of beta_OLS will be larger than it is beta_GLS? – Michael Webb Dec 1 '20 at 16:12
• @MichaelWebb : Gauss-Markov refers to the variance on a population level, i.e. under heteroskedasticity $\beta_{OLS}$ will "wiggle" more than $\beta_{GLS}$ when looking at multiple finite samples. The sample variance (or se) of $\beta_{OLS}$ need not be underestimated if correctly estimated via heteroskedasticity-robust var-cov matrices. The "usual" denominator of the t-test statistic implies homosk. In case of heterosk, we need to use a sandwich-type var-cov matrix. See vcovHC() from the R package sandwich. – PaulG Dec 3 '20 at 17:01
• @MichaelWebb : So yes, Gauss-Markov implies that, on average, the sample variance of the OLSE will be larger than the one for the GLSE, since this is true for the population variances (it's also how you'd simulate this). Note that the GLS estimator is not usable in practice, since $\Omega$ is unknown. Instead a feasible GLS (FGLS) or OLS with heterosk-robust errors is used. The confidence interval is a 'reversal' of the test statistic. A 100% coverage would mean $(-\infty; + \infty)$. If homosk is wrongly assumed for OLSE, then the confidence interval will tend to be too small. – PaulG Dec 3 '20 at 17:11

As regards the causes of heteroskedasticity in an econometrics setting:

In a regression setting and in econometrics, a fundamental distinction must be made between "conditional heteroskedasticity" (meaning, being a function of the regressors), and unconditional heteroskedasticity (meaning not being a function of the regressors).

We are talking about a property of the "error" term. While this is a terminology linked to the view of a regression relation as an approximation, in econometrics this is not so. In econometrics, regressions represent causal relationships between variables, where causality is argued from theory, past experience and logic (when the causal relations reflected in a specific regression are incomplete, then we have issues as to whether we can actually estimate the causal relationship accurately, but that is another matter).

Given this, the "error" term represents *all other factors that affect the dependent variable". And these "other factors" rarely are some "out of nowhere" "random shocks", some lottery to make life more interesting. And so here a "proportionality argument" kicks in:
Suppose that we examine a sample of production data of firms in the same industry. We expect to see firms of many different "sizes" (measured by output level, inputs levels). Say, firm $$k$$ is 10 times larger than firm $$j$$ in this sample. Is it reasonable to assume that the "error" related to firm $$k$$ has the same variance as the "error" related to the $$j$$ variable? Namely that it has the same "usual variation around its mean"?

If this was the case, then "all other factors" would affect disproportionately more the output of the small $$j$$ firm than the output of the big $$k$$ firm. This does not seem a very realistic assumption to make. Rather we anticipate that with each "size" goes an analogous "strength" of "all other factors" that affect the dependent variable. (In my language a proverb exists roughly translated as: "Big ship, big storms"). Big firms exert much more influence on their socioeconomic environment and this environment takes notice and attempts to influence back the big firms.

To mend this, the concept of "heteroskedasticity conditional on the regressors", was born. The higher the level of the regressors (say, production inputs), the higher the variance of the "error", to provide a degree of proportionality between the two groups of forces (regressors and "error") that determine the dependent variable.

In light of the above, what interpretation could we give to, and what usage could we have for unconditional heteroskedasticity (meaning, heteroskedasticity independent of the regressors)? Well, it is a convenient way to model a "summary" heteroskedasticity situation. Essentially, it is not trully "unconditional" in general, just independent of the regressors. In econometrics, such unconditional heteroskedasticity is essentially behind the concepts of "group-wise heteroskedasticity" which has evolved into what is nowadays called "clustered standard errors". Here, we assume heteroskedasticity not observation-per-observation, but between groups of observations (clusters), where group-membership is usually determined by a single variable which often is not part of the theoretically postulated regressors, but on some other aspect of the entities that are sampled (and so it would be part of "all other factors affecting the dependent variable").