0
$\begingroup$

I am getting confused about the derivation of standard errors for the OLS estimator $\widehat{\beta}$. I have seen two different ways to derive standard errors: (i) from the exact covariance matrix of $\widehat{\beta}$ conditional on the regressors' matrix $X$ and (ii) the approximation from the asymptotic distribution of $\widehat{\beta}$. While the derivation from the asymptotic distribution makes perfectly sense to me, the derivation from the conditional covariance matrix is unclear. I understand that: $$\text{Var}\left[\widehat{\beta} | X \right] = \left(X'X \right)^{-1} \text{E}\left[uu' | X \right] \left(X'X \right)^{-1},$$ where $\text{E}\left[uu' | X \right] = \text{Var}\left(u | X \right)$, i.e. the covariance matrix of the error terms that is generally a function of $X$ (heteroskedasticity). Then, my textbook claims that by replacing each element $i$ on the diagonal of $\text{E}\left[uu' | X \right]$ with the residuals squared $\widehat{u}^2_i$ we obtain a consistent estimate of the covariance matrix of $\widehat{\beta}$. However, from what I understand what we obtained is just an estimate of the variance of $\widehat{\beta}$ conditional on $X$, that still depends on $X$, hence this tells nothing about the unconditional covariance matrix of the estimator.

On the other hand, when we estimate asymptotic standard errors, we are able to approximate the unconditional covariance matrix of the OLS estimator. From what I understand it makes no sense to talk about "conditional standard errors", since in practical applications we are concerned about "unconditional standard errors".

$\endgroup$

2 Answers 2

2
$\begingroup$

The variance of $\widehat{\beta} $ is always conditional on $X$ (regressors) regardless of homo/heteroscedasticity and/or exact or asymptotically distributed errors. Sometimes the conditioning for $X$ is implicit but it is still used.

Moreover under homoscedasticity the form you have written simplifies to:

$$\text{Var}\left[\widehat{\beta} | X \right] = \left(X'X \right)^{-1} \text{E}\left[uu' | X \right] \left(X'X \right)^{-1}= \sigma^2 \left(X'X \right)^{-1}$$

But regardless of this simplification the conditioning on $X$ remains. Asymptotically, the formulas become a bit different but this does not matters to conclusions about conditioning.

Under homoscedasticity the unconditional variance looks like:

$$E[\text{Var}\left[\widehat{\beta} | X \right] ]= \sigma^2 E[\left(X'X \right)^{-1}] $$

but it is not known and is usually not considered.

$\endgroup$
2
  • $\begingroup$ Thanks for the reply. Thus, when considering the asymptotic distribution of $\widehat{\beta}$ what we are really talking about is the conditional asymptotic distribution? Meaning that with "asymptotic variance" of $\widehat{\beta}$ we refer to its conditional asymptotic variance? $\endgroup$ Commented Feb 11 at 14:23
  • 1
    $\begingroup$ Yes, we refer on that. $\endgroup$
    – markowitz
    Commented Feb 11 at 15:32
3
$\begingroup$

You say since in practical applications we are concerned about "unconditional standard errors", but I don't think this is true. Conditional standard errors tell us how uncertain we are about $\beta$, considering the $X$s we have. Unconditional standard errors tell us how uncertain we would expect to be about $\beta$ before we know what $X$s we have. Since we typically do know what $X$s we have, the conditional standard errors are preferable. They are also a lot easier to estimate; we try to avoid estimating the distribution of $X$ if we can, since it's multivariate and quite likely complicated.

You need unconditional (or expected conditional) standard errors for power/sample-size calculations in observational studies (in designed experiments you probably do know at least some components of $X$ in advance)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.