I have to solve the following problem: I've been given that $E[y_i|x_i] $ is non-linear in $ x_i = (1, x_i1, . . . , x_ik) $ (I have not been told "how" is non linear) and $ Var[y_i|x_i] = σ^2 $. I've to prove that via linear model approximation of the Conditional expectation function, the difference between the regression line and the Conditional expectation function depends only on $x_i$. Substantially, I need to show that the variance $ u_i $ is heteroskedastic by construction. Also, as a hint I've been told to start from $ E[(y_i − x_iβ)^2|x_i]$ and rewrite is as a function of $ E[y_i^2|x_i]$ plus something else.

I noticed that the term $ (y_i − x_iβ)^2$ is the square of the residual, which I further decomposed in $E[y_i^2|x_i]$ plus $E[x_iβ(x_iβ-2y_i)|x_i]$. Then I basically got stuck, because I don't know how to further proceed. I've noticed, and for sure I'm wrong, that $ Var[y_i|x_i] = σ^2 $ which means that the conditional variance takes a constant value, whereas $y_i$, in order to be heteroscedastic, should have a variance which change with i. It quite reminds me the homoskedasticity assumption, rather that the heteroskedastic one. But what I need to prove is that the error is heteroskedastic. How can I proceed in order to prove it?

  • $\begingroup$ We're to assume the variance is constant but that $u_i$ is heteroskedastic? This is apparently a contradiction. I'm guessing that it's the error term, but what is $u_i$? $\endgroup$
    – dsaxton
    Commented Mar 14, 2016 at 19:13
  • $\begingroup$ indeed! It looks like a contradiction! $u_i$ is the error term of the regression for which I have to prove the heteroskedasticity following the above steps. $\endgroup$
    – user108599
    Commented Mar 14, 2016 at 21:38

1 Answer 1


By construction

$$y_i = E(y_i\mid \mathbf x_i) + e_i,\;\;\;\; E(e_i\mid \mathbf x_i) = 0$$

We are told that $E(y_i\mid \mathbf x_i) = h(\mathbf x_i)$ and non-linear. Apply a first-order Taylor expansion to $h(\mathbf x_i)$, around a fixed value, say $E(\mathbf x_i)$:

$$h(\mathbf x_i) = h[E(\mathbf x_i)] + \nabla_xh[E(\mathbf x_i)]'[\mathbf x_i - E(\mathbf x_i)] + R_{1i}(\mathbf x_i - E(\mathbf x_i)) $$

where $R_{1i}(\mathbf x_i - E(\mathbf x_i))$ is the remainder, dependning on $\mathbf x_i$. For clarity we will henceforth write simply $R_{1i}$. Rearrange,

$$h(\mathbf x_i) = \Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) + \nabla_xh[E(\mathbf x_i)]'\mathbf x_i + R_{1i})$$

Notice that the term in the big parenthesis is a constant, and so is $\nabla_xh[E(\mathbf x_i)]$, since it is evaluated at the expected value. Map

$$\Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) \equiv \alpha_0,\;\;\; \nabla_xh[E(\mathbf x_i)] \equiv \beta $$

to obtain

$$h(\mathbf x_i) = \alpha_0 + \mathbf x_i'\beta + R_{1i}$$

Specify a linear regression equation

$$y_i = \alpha_0 + \mathbf x_i'\beta + u_i$$

It follows that

$$u_i = R_{1i} + e_i$$

We can now prove that $u_i$ is conditionally homoskedastic.

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}\left(R_{1i}\mid \mathbf x_i\right) + \text{Var}(e_i \mid \mathbf x_i) +2 \text{Cov}(R_{1i},e_i \mid \mathbf x_i)$$

But conditional on $\mathbf x_i$, $R_{1i}$ is fixed so its conditional variance is zero. Moreover,

$$\text{Cov}(R_{1i},e_i \mid \mathbf x_i) = E(R_{1i}e_i \mid \mathbf x_i) - E(R_{1i} \mid \mathbf x_i)E(e_i \mid \mathbf x_i) = R_{1i}E(e_i \mid \mathbf x_i)) - 0 = 0$$.

So we arrive at

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}(e_i \mid \mathbf x_i) = \text{Var}(y_i \mid \mathbf x_i)=\sigma^2$$

What does hold is that the our linear regression error term is mean-dependent on the regressors, because

$$u_i = R_{1i} + e_i \implies E(u_i \mid \mathbf x_i) = E(R_{1i} \mid \mathbf x_i) + E(e_i \mid \mathbf x_i) \implies E(u_i \mid \mathbf x_i) = R_{1i}.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.