# Derivation of OLS Variance

I stumbled over a rather simple result of OLS regression which is

$$Var[\hat\beta] = \sigma^2(X^TX)^{-1}$$ where $\sigma$ is the variance of the error term $u$ and $X$ is the regressor matrix.

I first just accepted the proof in my textbook but now I am thinking that it either uses sloppy notation or I am missing something. $\hat\beta$ is the estimated and $\beta$ is the true parameter (assuming unbiasedness).

It states that

\begin{align} Var[\hat\beta] &= E[(\hat\beta - \beta)(\hat\beta-\beta)^T] \\ &= E[(X^TX)^{-1}X^Tuu^TX(X^TX)^{-1}] \\ &= (X^TX)^{-1}X^T E[uu^T] X(X^TX)^{-1} \end{align}

but $X$ was only assumed to be exogenous and not non-stochastic. Under this assumption I think $X$ cannot be dragged outside the expectation operator.

Momentarily, I think that it should be $Var[\hat\beta|X]$ to make sense. Is that the case? My web research couldn't clarify this. I only found similar derivations to the above without further explainations.

• This looks like frequentist statistics and thus data or observations should be random, not the inputs or regressors, shouldn't they? The data are random because there is random error added. – gwr Nov 28 '15 at 11:40
• A couple of pages before this derivation the book authors state that assuming $X$ as nonstochastic is "frequently not a reasonable assumption" . Therfore, this assumption was relaxed to exogeneity. – Alex Nov 28 '15 at 11:46
• In section 6 of this tutorial it gives that the simplification only holds if $X$ is assumed non-stochastic. – gwr Nov 28 '15 at 12:03
• This would support my assumption that if we assume only exogeneity the correct notation would have been $Var[\hat\beta|X]$ and not $Var[\hat\beta]$. But why is this fact skipped so often? Seems not correct to me. – Alex Nov 28 '15 at 13:42
• It seems that under the assumptions usually made the usual estimate for the variance given that $X$ is fixed is also an unbiased estimate of the unconditional estimate when $X$ is random as is shown here. Maybe that helps. – gwr Nov 28 '15 at 15:14

You are right that the conditional variance is not generally the same as the unconditional one. By the variance decomposition lemma, which says that, for r.v.s $X$ and $Y$
$$Var(X)=E[Var(X|Y)]+Var[E(X|Y)]$$ Translated to our problem, $$Var(\widehat{\beta})=E[Var(\widehat{\beta}|X)]+Var[E(\widehat{\beta}|X)]$$ Now, using that OLS is conditionally unbiased (under suitable assumptions like exogeneity assumed here), we have $$E(\widehat{\beta}|X)=\beta$$ and thus $$Var[E(\widehat{\beta}|X)]=0,$$ as $\beta$ is a constant, so that $$Var(\widehat{\beta})=E[Var(\widehat{\beta}|X)]$$ or $$Var(\widehat{\beta})=E[\sigma^2(X'X)^{-1}].$$