# How to prove the Heteroskedasticiy of a $u_i$ in a linear regression model

I have to solve the following problem: I've been given that $E[y_i|x_i]$ is non-linear in $x_i = (1, x_i1, . . . , x_ik)$ (I have not been told "how" is non linear) and $Var[y_i|x_i] = σ^2$. I've to prove that via linear model approximation of the Conditional expectation function, the difference between the regression line and the Conditional expectation function depends only on $x_i$. Substantially, I need to show that the variance $u_i$ is heteroskedastic by construction. Also, as a hint I've been told to start from $E[(y_i − x_iβ)^2|x_i]$ and rewrite is as a function of $E[y_i^2|x_i]$ plus something else.

I noticed that the term $(y_i − x_iβ)^2$ is the square of the residual, which I further decomposed in $E[y_i^2|x_i]$ plus $E[x_iβ(x_iβ-2y_i)|x_i]$. Then I basically got stuck, because I don't know how to further proceed. I've noticed, and for sure I'm wrong, that $Var[y_i|x_i] = σ^2$ which means that the conditional variance takes a constant value, whereas $y_i$, in order to be heteroscedastic, should have a variance which change with i. It quite reminds me the homoskedasticity assumption, rather that the heteroskedastic one. But what I need to prove is that the error is heteroskedastic. How can I proceed in order to prove it?

• We're to assume the variance is constant but that $u_i$ is heteroskedastic? This is apparently a contradiction. I'm guessing that it's the error term, but what is $u_i$? – dsaxton Mar 14 '16 at 19:13
• indeed! It looks like a contradiction! $u_i$ is the error term of the regression for which I have to prove the heteroskedasticity following the above steps. – user108599 Mar 14 '16 at 21:38

By construction

$$y_i = E(y_i\mid \mathbf x_i) + e_i,\;\;\;\; E(e_i\mid \mathbf x_i) = 0$$

We are told that $$E(y_i\mid \mathbf x_i) = h(\mathbf x_i)$$ and non-linear. Apply a first-order Taylor expansion to $$h(\mathbf x_i)$$, around a fixed value, say $$E(\mathbf x_i)$$:

$$h(\mathbf x_i) = h[E(\mathbf x_i)] + \nabla_xh[E(\mathbf x_i)]'[\mathbf x_i - E(\mathbf x_i)] + R_{1i}(\mathbf x_i - E(\mathbf x_i))$$

where $$R_{1i}(\mathbf x_i - E(\mathbf x_i))$$ is the remainder, dependning on $$\mathbf x_i$$. For clarity we will henceforth write simply $$R_{1i}$$. Rearrange,

$$h(\mathbf x_i) = \Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) + \nabla_xh[E(\mathbf x_i)]'\mathbf x_i + R_{1i})$$

Notice that the term in the big parenthesis is a constant, and so is $$\nabla_xh[E(\mathbf x_i)]$$, since it is evaluated at the expected value. Map

$$\Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) \equiv \alpha_0,\;\;\; \nabla_xh[E(\mathbf x_i)] \equiv \beta$$

to obtain

$$h(\mathbf x_i) = \alpha_0 + \mathbf x_i'\beta + R_{1i}$$

Specify a linear regression equation

$$y_i = \alpha_0 + \mathbf x_i'\beta + u_i$$

It follows that

$$u_i = R_{1i} + e_i$$

We can now prove that

A. $$u_i$$ is conditionally homoskedastic and

B. $$u_i$$ is unconditionally heteroskedastic

A. Conditional Homoskedasticity

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}\left(R_{1i}\mid \mathbf x_i\right) + \text{Var}(e_i \mid \mathbf x_i) +2 \text{Cov}(R_{1i},e_i \mid \mathbf x_i)$$

But conditional on $$\mathbf x_i$$, $$R_{1i}$$ is fixed so its conditional variance is zero. Moreover,

$$\text{Cov}(R_{1i},e_i \mid \mathbf x_i) = E(R_{1i}e_i \mid \mathbf x_i) - E(R_{1i} \mid \mathbf x_i)E(e_i \mid \mathbf x_i) = R_{1i}E(e_i \mid \mathbf x_i)) - 0 = 0$$.

So we arrive at

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}(e_i \mid \mathbf x_i) = \text{Var}(y_i \mid \mathbf x_i)=\sigma^2$$

B. Unconditional Heteroskedasticity

By the Law of Total Variance $$\text{Var}(u_i) = E\big[\text{Var}(u_i \mid \mathbf x_i)\big] + \text{Var}\big[E(u_i\mid \mathbf x_i)\big]$$

$$=E\big[\sigma^2\big] + \text{Var}\big[E(R_{1i}\mid \mathbf x_i)+E(e_i\mid \mathbf x_i)\big]$$

$$= \sigma^2 + \text{Var}\big[R_{1i}E(1\mid \mathbf x_i) + 0\big]$$

$$\implies \text{Var}(u_i) = \sigma^2 + \text{Var}\big[R_{1i}\big]$$

What is interesting to realize here is that if the linear regression specification is just an approximation to the true conditional expectation we obtain a situation that is the reverse of the "usual" set up: instead of having an error term conditionally heteroskedatic but unconditionally homoskedastic, we have that the error term of our specification is conditionally homoskedastic but unconditionally heteroskedastic.

While finite-sample inference is usually thought of as "conditional on the regressors", the above result will affect any inference based on/justified by asymptotic results, where unconditional moments come into play.