Consider the following true data generating process (DGP):

$$y = x_1\beta_1 + \ldots + x_p \beta_p + \varepsilon = x_{true}' \beta + \varepsilon$$

where $x_i \in {R}^n$, $y \in {R}^n$ and ${E}(\varepsilon) = 0$, $Var(\varepsilon) = \sigma^2$. The corresponding OLS estimator is denoted by $\hat{\beta}_{true}$.

Underfitting: the model is estimated with less variables than needed. I.e. $q<p$ variables are used: $$y = x_1\beta_1 + \ldots + x_q \beta_q + \varepsilon = x_{under}' \beta + \varepsilon$$

My aim is to prove that in this case, the OLS prediction is biased but with smaller variance.

\begin{eqnarray*} {E}(x'_{under}\hat{\beta}_{under}) \neq x_{true}'\beta \quad & & Var(x_{under}'\hat{\beta}_{under}) \leq Var(x_{true}'\hat{\beta}_{true}) \end{eqnarray*}

$ $\

Progress: I have been doing progress in the expected value. If the true model is the next:$$Y=X_{True}\beta_{True}+\epsilon$$, and in the underfitting model we use less variables, we have that: $$Y=X_{under}\beta_{under}+\epsilon$$. We can observe that: $$\hat{\beta}_{True} = (X_{True}^T X_{True})^{-1}X_{True}^T Y$$ $$\hat{\beta}_{under} = (X_{under}^T X_{under})^{-1}X_{under}^T Y$$ so $$E(\hat{\beta}_{True})=\beta_{True}$$ and $$E(\hat{\beta}_{under})=\beta_{under}$$. So they are different, ¿Is this correct?

  • $\begingroup$ You should add the self-study tag, and then give some more information. You must define all your terms! $\endgroup$ – kjetil b halvorsen Jun 5 '16 at 14:04
  • $\begingroup$ I have m ade the question more clear, l and tried to do it in my own. but unfortunatelly I m not capable :( @ kjetil b halvorsen $\endgroup$ – Mike13 Jun 5 '16 at 16:41
  • 1
    $\begingroup$ What do you specifically mean by $\text{Var}(x'_{\mathrm{True}} \hat{\beta}_{\mathrm{True}})$? What is $x_{\mathrm{True}}$ here? Are you treating the $x_{\mathrm{True}}$ as a random variable? Or as exogenous? A specific observation? an average of the observations? I don't follow. $\endgroup$ – Matthew Gunn Jun 7 '16 at 22:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.