How to prove the Heteroskedasticiy of a $ u_i $ in a linear regression model

Question

I have to solve the following problem: I've been given that $E[y_i|x_i] $ is non-linear in $ x_i = (1, x_i1, . . . , x_ik) $ (I have not been told "how" is non linear) and $ Var[y_i|x_i] = σ^2 $. I've to prove that via linear model approximation of the Conditional expectation function, the difference between the regression line and the Conditional expectation function depends only on $x_i$. Substantially, I need to show that the variance $ u_i $ is heteroskedastic by construction. Also, as a hint I've been told to start from $ E[(y_i − x_iβ)^2|x_i]$ and rewrite is as a function of $ E[y_i^2|x_i]$ plus something else.

I noticed that the term $ (y_i − x_iβ)^2$ is the square of the residual, which I further decomposed in $E[y_i^2|x_i]$ plus $E[x_iβ(x_iβ-2y_i)|x_i]$. Then I basically got stuck, because I don't know how to further proceed. I've noticed, and for sure I'm wrong, that $ Var[y_i|x_i] = σ^2 $ which means that the conditional variance takes a constant value, whereas $y_i$, in order to be heteroscedastic, should have a variance which change with i. It quite reminds me the homoskedasticity assumption, rather that the heteroskedastic one. But what I need to prove is that the error is heteroskedastic. How can I proceed in order to prove it?

Answer 1

By construction

$$y_i = E(y_i\mid \mathbf x_i) + e_i,\;\;\;\; E(e_i\mid \mathbf x_i) = 0$$

We are told that $E(y_i\mid \mathbf x_i) = h(\mathbf x_i)$ and non-linear. Apply a first-order Taylor expansion to $h(\mathbf x_i)$, around a fixed value, say $E(\mathbf x_i)$:

$$h(\mathbf x_i) = h[E(\mathbf x_i)] + \nabla_xh[E(\mathbf x_i)]'[\mathbf x_i - E(\mathbf x_i)] + R_{1i}(\mathbf x_i - E(\mathbf x_i)) $$

where $R_{1i}(\mathbf x_i - E(\mathbf x_i))$ is the remainder, dependning on $\mathbf x_i$. For clarity we will henceforth write simply $R_{1i}$. Rearrange,

$$h(\mathbf x_i) = \Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) + \nabla_xh[E(\mathbf x_i)]'\mathbf x_i + R_{1i})$$

Notice that the term in the big parenthesis is a constant, and so is $\nabla_xh[E(\mathbf x_i)]$, since it is evaluated at the expected value. Map

$$\Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) \equiv \alpha_0,\;\;\; \nabla_xh[E(\mathbf x_i)] \equiv \beta $$

to obtain

$$h(\mathbf x_i) = \alpha_0 + \mathbf x_i'\beta + R_{1i}$$

Specify a linear regression equation

$$y_i = \alpha_0 + \mathbf x_i'\beta + u_i$$

It follows that

$$u_i = R_{1i} + e_i$$

We can now prove that $u_i$ is conditionally homoskedastic.

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}\left(R_{1i}\mid \mathbf x_i\right) + \text{Var}(e_i \mid \mathbf x_i) +2 \text{Cov}(R_{1i},e_i \mid \mathbf x_i)$$

But conditional on $\mathbf x_i$, $R_{1i}$ is fixed so its conditional variance is zero. Moreover,

$$\text{Cov}(R_{1i},e_i \mid \mathbf x_i) = E(R_{1i}e_i \mid \mathbf x_i) - E(R_{1i} \mid \mathbf x_i)E(e_i \mid \mathbf x_i) = R_{1i}E(e_i \mid \mathbf x_i)) - 0 = 0$$.

So we arrive at

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}(e_i \mid \mathbf x_i) = \text{Var}(y_i \mid \mathbf x_i)=\sigma^2$$

What does hold is that the our linear regression error term is mean-dependent on the regressors, because

$$u_i = R_{1i} + e_i \implies E(u_i \mid \mathbf x_i) = E(R_{1i} \mid \mathbf x_i) + E(e_i \mid \mathbf x_i) \implies E(u_i \mid \mathbf x_i) = R_{1i}.$$