I am reading Wooldridge's Introductory Econometrics (2000), don't judge me, old version = cheap second hand book, and in the page P94 Theorem 3.2 of Multiple Regression Analysis, it says that:

$$ Var(\hat{\beta_{j}}) = \dfrac{\sigma^2}{SST_j(1-R^2_j)}\tag{1} $$

where $SST_j$ is the total sample variation in $x_j$, $R^2_j$ is the $R$-squared from regressing $x_j$ on all other independent variables. I do understanding the meaning of this formula, and it has a algebraic proof in appendix. So this equation, along with the expected value of $\beta_j$ shows several properties such as: variance of betas in misspeficied models, and also why adding irrelevant independent variables in regression model to boost $R$-squared is at the cost of high variance of OLS estimators.

But if we take a look at the same formula in matrix form, for instance, in Hayashi (2000), the same variance of $\hat{\beta}$ is:

$$ Var(\hat{\beta}) = \sigma^2(X'X)^{-1}\tag{2} $$

Now, the question is that how to show the above question in the following form:

$$ Var(\hat{\beta_{j}}) = \dfrac{\sigma^2}{nS^2_j(1-R^2_j)}\tag{3} $$

where $R_j$ is the $R$-squared from regressing $x_j$ on all other $x$’s, $n$ is the sample size and $S^2_j$ is the sample variance of the regressor $X$.

  • $\begingroup$ I'm fairly certain the $R_j$ you have listed in (3) is a centered $R^2$? $\endgroup$ – StatsStudent Jan 5 at 16:55
  • $\begingroup$ Got it...yes it is a centered r squared...but I was wondering since the text just mentioned that by doing some operation, we can get (3) from (2)...I was wondering how.. $\endgroup$ – commentallez-vous Jan 5 at 17:12
  • $\begingroup$ Well, you have to understand that (2) is a covariance matrix of the vector $\beta$ whereas (3) is just an individual variance of a specific estimated coefficient $\beta_j$. You'd essentially be working with the $j$-th diagonal element of the matrix in (2) (variances). $\endgroup$ – StatsStudent Jan 5 at 17:24
  • $\begingroup$ But (2) is the inverse of the var-cov matrix X'X, so I have no idea about the properties or what the inverse of X'X is doing here. $\endgroup$ – commentallez-vous Jan 5 at 17:28
  • $\begingroup$ Do you have much knowledge in matrix algebra (I just don't have a good sense of your level here). If not, I'd highly recommend you check out chapter 5 of Applied Linear Statistical Models Fifth Edition by Kutner et al. for a good crash course. I think this will also many of your questions. If I have time later, I'll try to work this out for you and write up a complete answer. $\endgroup$ – StatsStudent Jan 5 at 17:38

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.