I have to solve the following problem: I've been given that $E[y_i|x_i] $ is non-linear in $ x_i = (1, x_i1, . . . , x_ik) $ (I have not been told "how" is non linear) and $ Var[y_i|x_i] = σ^2 $. I've to prove that via linear model approximation of the Conditional expectation function, the difference between the regression line and the Conditional expectation function depends only on $x_i$. Substantially, I need to show that the variance $ u_i $ is heteroskedastic by construction. Also, as a hint I've been told to start from $ E[(y_i − x_iβ)^2|x_i]$ and rewrite is as a function of $ E[y_i^2|x_i]$ plus something else.

I noticed that the term $ (y_i − x_iβ)^2$ is the square of the residual, which I further decomposed in $E[y_i^2|x_i]$ plus $E[x_iβ(x_iβ-2y_i)|x_i]$. Then I basically got stuck, because I don't know how to further proceed. I've noticed, and for sure I'm wrong, that $ Var[y_i|x_i] = σ^2 $ which means that the conditional variance takes a constant value, whereas $y_i$, in order to be heteroscedastic, should have a variance which change with i. It quite reminds me the homoskedasticity assumption, rather that the heteroskedastic one. But what I need to prove is that the error is heteroskedastic. How can I proceed in order to prove it?

  • $\begingroup$ We're to assume the variance is constant but that $u_i$ is heteroskedastic? This is apparently a contradiction. I'm guessing that it's the error term, but what is $u_i$? $\endgroup$ – dsaxton Mar 14 '16 at 19:13
  • $\begingroup$ indeed! It looks like a contradiction! $u_i$ is the error term of the regression for which I have to prove the heteroskedasticity following the above steps. $\endgroup$ – user108599 Mar 14 '16 at 21:38

By construction

$$y_i = E(y_i\mid \mathbf x_i) + e_i,\;\;\;\; E(e_i\mid \mathbf x_i) = 0$$

We are told that $E(y_i\mid \mathbf x_i) = h(\mathbf x_i)$ and non-linear. Apply a first-order Taylor expansion to $h(\mathbf x_i)$, around a fixed value, say $E(\mathbf x_i)$:

$$h(\mathbf x_i) = h[E(\mathbf x_i)] + \nabla_xh[E(\mathbf x_i)]'[\mathbf x_i - E(\mathbf x_i)] + R_{1i}(\mathbf x_i - E(\mathbf x_i)) $$

where $R_{1i}(\mathbf x_i - E(\mathbf x_i))$ is the remainder, dependning on $\mathbf x_i$. For clarity we will henceforth write simply $R_{1i}$. Rearrange,

$$h(\mathbf x_i) = \Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) + \nabla_xh[E(\mathbf x_i)]'\mathbf x_i + R_{1i})$$

Notice that the term in the big parenthesis is a constant, and so is $\nabla_xh[E(\mathbf x_i)]$, since it is evaluated at the expected value. Map

$$\Big(h[E(\mathbf x_i)] -\nabla_xh[E(\mathbf x_i)]'E(\mathbf x_i)\Big) \equiv \alpha_0,\;\;\; \nabla_xh[E(\mathbf x_i)] \equiv \beta $$

to obtain

$$h(\mathbf x_i) = \alpha_0 + \mathbf x_i'\beta + R_{1i}$$

Specify a linear regression equation

$$y_i = \alpha_0 + \mathbf x_i'\beta + u_i$$

It follows that

$$u_i = R_{1i} + e_i$$

We can now prove that

A. $u_i$ is conditionally homoskedastic and

B. $u_i$ is unconditionally heteroskedastic

A. Conditional Homoskedasticity

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}\left(R_{1i}\mid \mathbf x_i\right) + \text{Var}(e_i \mid \mathbf x_i) +2 \text{Cov}(R_{1i},e_i \mid \mathbf x_i)$$

But conditional on $\mathbf x_i$, $R_{1i}$ is fixed so its conditional variance is zero. Moreover,

$$\text{Cov}(R_{1i},e_i \mid \mathbf x_i) = E(R_{1i}e_i \mid \mathbf x_i) - E(R_{1i} \mid \mathbf x_i)E(e_i \mid \mathbf x_i) = R_{1i}E(e_i \mid \mathbf x_i)) - 0 = 0$$.

So we arrive at

$$\text{Var}(u_i \mid \mathbf x_i) = \text{Var}(e_i \mid \mathbf x_i) = \text{Var}(y_i \mid \mathbf x_i)=\sigma^2$$

B. Unconditional Heteroskedasticity

By the Law of Total Variance $$\text{Var}(u_i) = E\big[\text{Var}(u_i \mid \mathbf x_i)\big] + \text{Var}\big[E(u_i\mid \mathbf x_i)\big]$$

$$=E\big[\sigma^2\big] + \text{Var}\big[E(R_{1i}\mid \mathbf x_i)+E(e_i\mid \mathbf x_i)\big]$$

$$= \sigma^2 + \text{Var}\big[R_{1i}E(1\mid \mathbf x_i) + 0\big]$$

$$\implies \text{Var}(u_i) = \sigma^2 + \text{Var}\big[R_{1i}\big] $$

What is interesting to realize here is that if the linear regression specification is just an approximation to the true conditional expectation we obtain a situation that is the reverse of the "usual" set up: instead of having an error term conditionally heteroskedatic but unconditionally homoskedastic, we have that the error term of our specification is conditionally homoskedastic but unconditionally heteroskedastic.

While finite-sample inference is usually thought of as "conditional on the regressors", the above result will affect any inference based on/justified by asymptotic results, where unconditional moments come into play.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.