2
$\begingroup$

(I don't remember seeing this result stressed enough.)

Consider the "benchmark" linear regression model

$$\mathbf{Y} = \mathbf{X} \beta + \varepsilon.$$

$$E(\varepsilon) = 0,\, E(\varepsilon \varepsilon' \mid \mathbf X) = \sigma^2I.$$

The initial presentation of this model in econometrics, is with the additional assumption of "strict exogeneity", $E(\varepsilon \mid \mathbf X) =0$. But usually, we move on to say that this assumption is too strong and not expected to hold, and we settle for the weaker assumption $E(\mathbf X' \varepsilon) = 0$. With this assumption, we lose finite-sample unbiasedness of the OLS estimator, but we retain consistency and asymptotic normality.

Now, in order for the weaker model to be usefully discriminated from the model with strict exogeneity, it must be the case that we have (alongside $E(\mathbf X' \varepsilon) = 0$), $E(\varepsilon \mid \mathbf X) =h(\mathbf X)$ also.

Question: in such a setting, what does the variance estimator of $\hat \beta_{OLS}$ estimate?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

By definition, under any case the finite-sample variance of $\hat \beta$

is

$${\rm V}(\hat \beta \mid \mathbf X) = (\mathbf X'\mathbf X)^{-1}\mathbf X'E\left[\varepsilon \varepsilon' \mid \mathbf X\right]\mathbf X(\mathbf X'\mathbf X)^{-1} $$ $$- (\mathbf X'\mathbf X)^{-1}\mathbf X'E\left[\varepsilon \mid \mathbf X\right]E\left[\varepsilon' \mid \mathbf X\right]\mathbf X(\mathbf X'\mathbf X)^{-1}.$$

Under the assumptions without strict exogeneity but with orthogonality,

$$E(\varepsilon \mid \mathbf X) = h(\mathbf X),\quad E(\mathbf X' \varepsilon) = 0,\quad E(\varepsilon \varepsilon' \mid \mathbf X) =\sigma^2I, $$

the finite sample variance becomes

$${\rm V}(\hat \beta \mid \mathbf X) = \sigma^2(\mathbf X'\mathbf X)^{-1} - (\mathbf X'\mathbf X)^{-1}\mathbf X'h\left(\mathbf X\right)h'\left(\mathbf X\right)\mathbf X(\mathbf X'\mathbf X)^{-1},$$

while the (conditional) bias of the estimator is

$${\rm B}(\hat \beta \mid \mathbf X) = E(\hat \beta \mid X) - \beta = (\mathbf X'\mathbf X)^{-1}\mathbf X'h\left(\mathbf X\right).$$

It follows that the Mean Squared Error of $\hat \beta$ is

$${\rm MSE}(\hat \beta \mid \mathbf X) = {\rm V}(\hat \beta \mid \mathbf X) + {\rm B}(\hat \beta \mid \mathbf X){\rm B}'(\hat \beta \mid \mathbf X) = \sigma^2\left(\mathbf X' \mathbf X\right)^{-1}.$$

We see that , when we do not assume strict exogeneity but only orthogonality, the finite sample "variance" estimator (that is automatically computed by software) in reality computes the finite-sample Mean Squared Error (this continues to hold if we assume conditional heteroskedasticity and want to compute "robust" standard errors).

In general when we expect the estimator to be biased, we tend to prefer using MSE as a more appropriate "quality" measure than the Variance alone. But this does not mean that we can use MSE in all cases instead of the Variance without consequences. For example, in significance testing, the value of the $t$-statistic will be deflated (in absolute terms) due to the use of the estimated MSE in its denominator.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Hi Alecos: Not only is it not stressed but I never even seen it or known that it was true !!!! One question, though. If one knows that $E(\epsilon | X) = h(X)$, then can't one just "transform all the predictor variables by subtracting off $h(X)$ and then be back to the exogenous case ? $\endgroup$
    – mlofton
    Dec 5, 2020 at 17:32
  • $\begingroup$ @mlofton We usually do not know $h(\mathbf X)$, we just argue structurally that there exists mean-dependence of unknown functional form. $\endgroup$
    – Alecos Papadopoulos
    Dec 5, 2020 at 17:38
  • $\begingroup$ Thanks. Do you know of any texts, stats or econometrics, that talk about the difference between the exogeneity assumption and the orthogonality assumption. I have many and I've never seen such a discussion but maybe I missed it ? $\endgroup$
    – mlofton
    Dec 7, 2020 at 0:09
  • $\begingroup$ @mlofton Well, it is standard practice in econometrics textbooks (eg. Greene, Hayashi) to initially present linear regression with strict exogeneity and talk about finite sample properties, and then discuss asymptotic properties, where they assume only orthogonality. But they don't contrast the two and they don't discuss what happens to finite-sample inference under orthogonality only. That was the reason for this post. $\endgroup$
    – Alecos Papadopoulos
    Dec 7, 2020 at 1:12
  • $\begingroup$ Thanks. I have hayashi so I'll check it out. That description seems better than what they do in statistics textbooks where they generally talk about orthogonality and that's that. Exogeneity is not discussed AFAIK. $\endgroup$
    – mlofton
    Dec 8, 2020 at 2:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.