Standard error of components of coefficient estimate in linear regression

Question

As I understand it, $\text{standard error} = \frac{\text{standard deviation}}{\sqrt{n}}$, where $n$ is the number of observations. I also know that $cov(\hat{\beta}) = \sigma^{2}(X^{T}X)^{-1}$; this is a pxp variance-covariance matrix. So why is the standard error of the components of the coefficient estimate vector $\boldsymbol{\hat{\beta}}$ in, for example, a normal linear model, $\sigma^{2}[(X^{T}X)^{-1}]_{ii}$ and not $\frac{1}{\sqrt{n}}\sigma^{2}[(X^{T}X)^{-1}]_{ii}$ where the subscript $ii$ denotes the $i,i$th entry of the matrix?

Answer 1

The "$n$" factor is implicit already in $\boldsymbol{X}^T\boldsymbol{X}$.

This is easiest to see in the case where $\boldsymbol{X}$ has been whitened, so $\boldsymbol{X}^T\boldsymbol{X}=n\boldsymbol{I}$.

More generally we have $\boldsymbol{X}=[\boldsymbol{x}_1,\ldots\boldsymbol{x}_p]$, where $\boldsymbol{x}_p\in\mathbb{R}^n$. Then the entries of the matrix are $$\left(\boldsymbol{X}^T\boldsymbol{X}\right)_{ij}=\boldsymbol{x}_i^T\boldsymbol{x}_j$$ Now choose a pair of columns, $\boldsymbol{a}=\boldsymbol{x}_i$ and $\boldsymbol{b}=\boldsymbol{x}_j$. Then we have $$\boldsymbol{a}^T\boldsymbol{b}=\sum_{k=1}^na_kb_k=\overline{(ab)}\,n$$ So each entry of the matrix is a dot product, which can be seen as $n$ times an average.

In the case where $\boldsymbol{X}$ has been centered so the columns have zero means, the product-matrix entries are covariances.

For an overview of the general case, see here.