Understanding the causes and implication of heteroskedasticity

Question

I’m trying to heteroskedasticity and how, even if we don’t have MLR 5 assumption (heteroskedasticity), we can still have unbiased estimates.

I was thinking: a very intuitive cause of a growing variance would be that we have an omitted variable bias. In this case, wouldn’t MLR 4 be violated as well, as thus the above statement is invalid?

(MLR 4 refers to conditional mean independence of X and error terms. I understand that this means that X and errors must be uncorrelated. Therefore an OVB would rule this out)

Am I just thinking of but one specific cause of heteroskedasticity, and thus the other causes can still preserve MLR 4? I understand another cause is autocorrelation, which is like inertia of an outcome with respect to an X variable, usually time (in time series).

Answer 1

Here is a source out of the econometric literature to summarize some possible reasons for heteroskedasticity which include:

1. As people learn the error of their behavior, as in typing mistakes, there can be an improvement (reduction in errors) over time.

2. As income grows people have more choices on the disposition of their income.

3. As data collection techniques (as in automated data processing) improve, associated data collection errors are reduced.

4. Outliers can be a factor in heteroskedasticity.

5. A model specification error can contribute to heteroskedasticity as occurs in the emission of important variables.

6. Skewness in the distribution of a regressor variable could be a possible source for heteroskedasticity as occurs, for example, in the distribution of wealth and income, where there is a significant concentration in a few.

7. In modeling, possible errors related to heteroskedasticity could occur relating to inappropriate data transformations.

8. Or inappropriate functional form as in linear versus a log-linear model.

9. And finally, heteroskedasticity is more common in cross-sectional data than conventional time series analysis.

I hope this helps.

Answer 2

I find the "MLR X" format somewhat confusing, so I won't use it here. They basically refer to the conditions under which the Gauss-Markov Theorem applies, i.e. the conditions under which the OLS estimator is BLUE (best linear unbiased estimator). The most important ones are: 1) Regressors must be exogenous, i.e. $E[u|X]=0$, and 2) errors must be homoskedastic, i.e. $Cov[u]=\sigma^2I$ where $I$ is the identity matrix. I tried to break down your post into 2 main questions:

Question 1: How can we have unbiased estimators with heteroskedasticity? Answer: unbiasedness has nothing to do with the variance and is not the result of the Gauss-Markov theorem. For unbiasedness it exogeneity suffices, i.e. $E[u|X]=0$. Proof: $$E[\hat \beta_{OLS}|X]=E[(X^TX)^{-1}X^Ty|X] \\ = E[(X^TX)^{-1}X^T(X\beta + u)|X] \\ =(X^TX)^{-1}(X^TX)\beta+ \underbrace{E[u|X]}_{=0}=\beta$$

Notice how the variance of $u$ is not imporant here, i.e. it could be that $Var[u_i]=\sigma^2_i$ with a different $\sigma^2_i$ each time (i.e. heteroskedasticity). Further notice that exogeneity implies no correlation, i.e. $u_i$ can not be autocorrelated.

Question 2: Can the Gauss-Makrov Theorem still hold under heteroskedasticity? Answer: Under heteroskedasticity, the Gauss-Markov Theorem does not apply anymore, i.e. the OLS estimator is not BLUE anymore. It still a LUE, i.e. linear and unbiased, but not best anymore. Here "best" refers to the estimator with the lowest variance. The GLS (generalized least squares) estimator is BLUE now instead, if $\Omega$ is any variance-covariance matrix, and not just $\Omega=\sigma^2 I$ anymore: $\beta_{GLS}=(X^T\Omega^{-1} X)^{-1}X^T \Omega^{-1}X$. Proof is somewhat lengthy, but not hard. Essentially it boils down to proving that $Var[\beta_{OLS}]-Var[\beta_{GLS}]\ge0$ for any $\Omega$.

Answer 3

As regards the causes of heteroskedasticity in an econometrics setting:

In a regression setting and in econometrics, a fundamental distinction must be made between "conditional heteroskedasticity" (meaning, being a function of the regressors), and unconditional heteroskedasticity (meaning not being a function of the regressors).

We are talking about a property of the "error" term. While this is a terminology linked to the view of a regression relation as an approximation, in econometrics this is not so. In econometrics, regressions represent causal relationships between variables, where causality is argued from theory, past experience and logic (when the causal relations reflected in a specific regression are incomplete, then we have issues as to whether we can actually estimate the causal relationship accurately, but that is another matter).

Given this, the "error" term represents *all other factors that affect the dependent variable". And these "other factors" rarely are some "out of nowhere" "random shocks", some lottery to make life more interesting. And so here a "proportionality argument" kicks in:
Suppose that we examine a sample of production data of firms in the same industry. We expect to see firms of many different "sizes" (measured by output level, inputs levels). Say, firm $k$ is 10 times larger than firm $j$ in this sample. Is it reasonable to assume that the "error" related to firm $k$ has the same variance as the "error" related to the $j$ variable? Namely that it has the same "usual variation around its mean"?

If this was the case, then "all other factors" would affect disproportionately more the output of the small $j$ firm than the output of the big $k$ firm. This does not seem a very realistic assumption to make. Rather we anticipate that with each "size" goes an analogous "strength" of "all other factors" that affect the dependent variable. (In my language a proverb exists roughly translated as: "Big ship, big storms"). Big firms exert much more influence on their socioeconomic environment and this environment takes notice and attempts to influence back the big firms.

To mend this, the concept of "heteroskedasticity conditional on the regressors", was born. The higher the level of the regressors (say, production inputs), the higher the variance of the "error", to provide a degree of proportionality between the two groups of forces (regressors and "error") that determine the dependent variable.

In light of the above, what interpretation could we give to, and what usage could we have for unconditional heteroskedasticity (meaning, heteroskedasticity independent of the regressors)? Well, it is a convenient way to model a "summary" heteroskedasticity situation. Essentially, it is not trully "unconditional" in general, just independent of the regressors. In econometrics, such unconditional heteroskedasticity is essentially behind the concepts of "group-wise heteroskedasticity" which has evolved into what is nowadays called "clustered standard errors". Here, we assume heteroskedasticity not observation-per-observation, but between groups of observations (clusters), where group-membership is usually determined by a single variable which often is not part of the theoretically postulated regressors, but on some other aspect of the entities that are sampled (and so it would be part of "all other factors affecting the dependent variable").

See also, https://stats.stackexchange.com/a/499049/28746