I was reading in "A Guide to Econometrics" that given $Y = X \beta + \epsilon$, the variance covariance matrix of $\beta^\text{OLS}$ is given by $\sigma^2 (X' X)^{-1}$ where $\sigma^2$ is the variance of the error term...

it then says that in the case of a single regressor $y = \beta_1 + \beta_2 x$, that this simplifies to $\sigma^2 / \sum(x-\bar{x})^2$. I don't quite see this. Isn't the matrix $X$ in this case still a two by two matrix? Namely:

$$ X = \left( \begin{array}{cc} 1 & x_1 \\\ 1 & x_2 \end{array} \right) $$

How are we getting something like $\sum(x-\bar{x})^2$ ?

  • $\begingroup$ Clearly the book was talking about the variance of the slope, i.e. $\beta_2$. The covariance of $(\beta_1,\beta_2)$ is still $2\times 2$ matrix as Charlie demonstrated. $\endgroup$ – mpiktas Sep 16 '11 at 6:56
  • $\begingroup$ Also $X$ is usually $n\times k$ matrix, with $n$ being number of observations, $k$ being the number of right hand side variables including the constant. In your case $k=2$. The matrix $(X'X)^{-1}$ is then usualy $k\times k$ matrix, in your case $2\times 2$. $\endgroup$ – mpiktas Sep 16 '11 at 6:58

Multiply $X^\prime X$: $$\begin{align*} \left[ \begin{array}{cc} 1^\prime 1 & 1^\prime x \\ x^\prime 1 & x^\prime x \end{array} \right] \end{align*}$$ Take the inverse (see here): $$\begin{align*} \frac{1}{Nx^\prime x - (1^\prime x)^2}\left[ \begin{array}{cc} x^\prime x & -1^\prime x \\ -1^\prime x & N \end{array} \right] \end{align*}$$

Note that $$\begin{align*}\sum{(x_i - \bar{x})^2} = \sum{x_i^2} - N \bar{x}^2.\end{align*}$$

Looking at the lower right corner of the matrix, we have $$\begin{align*}\frac{N}{N\sum{x_i^2} - \left(\sum{x_i}\right)^2}= \frac{1}{\sum{x_i^2} - \frac{1}{N}\left(\sum{x_i}\right)^2} = \frac{1}{\sum{x_i^2} - N \bar{x}^2} = \frac{1}{\sum{(x_i - \bar{x})^2}}.\end{align*}$$ Multiplying by $\sigma^2$ gives the variance of the slope coefficient.

  • $\begingroup$ aargh, I was composing my answer when notification came :) The third equation has an error by the way. $\endgroup$ – mpiktas Sep 16 '11 at 6:30
  • $\begingroup$ Hm I clearly saw the earlier version with an error, but SE probably merged the revision. Now everything is ok. The error was that $\frac{1}{N}$ was besides the first term in the right hand side. $\endgroup$ – mpiktas Sep 16 '11 at 6:52
  • 1
    $\begingroup$ (+1) The first formula (for $X'X$) still contains typos... (but the rest of the analysis is correct). $\endgroup$ – whuber Sep 16 '11 at 14:41
  • $\begingroup$ @Charlie Thanks a lot! That was crystal clear! I won't not have a notebook by my side the next time I read stats... $\endgroup$ – Palace Sep 17 '11 at 4:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.