I am reading Wooldridge's Introductory Econometrics (2000), don't judge me, old version = cheap second hand book, and in the page P94 Theorem 3.2 of Multiple Regression Analysis, it says that:

$$ Var(\hat{\beta_{j}}) = \dfrac{\sigma^2}{SST_j(1-R^2_j)}\tag{1} $$

where $SST_j$ is the total sample variation in $x_j$, $R^2_j$ is the $R$-squared from regressing $x_j$ on all other independent variables. I do understanding the meaning of this formula, and it has a algebraic proof in appendix. So this equation, along with the expected value of $\beta_j$ shows several properties such as: variance of betas in misspeficied models, and also why adding irrelevant independent variables in regression model to boost $R$-squared is at the cost of high variance of OLS estimators.

But if we take a look at the same formula in matrix form, for instance, in Hayashi (2000), the same variance of $\hat{\beta}$ is:

$$ Var(\hat{\beta}) = \sigma^2(X'X)^{-1}\tag{2} $$

Now, the question is that how to show the above question in the following form:

$$ Var(\hat{\beta_{j}}) = \dfrac{\sigma^2}{nS^2_j(1-R^2_j)}\tag{3} $$

where $R_j$ is the $R$-squared from regressing $x_j$ on all other $x$’s, $n$ is the sample size and $S^2_j$ is the sample variance of the regressor $X$.

  • $\begingroup$ I'm fairly certain the $R_j$ you have listed in (3) is a centered $R^2$? $\endgroup$ – StatsStudent Jan 5 at 16:55
  • $\begingroup$ Got it...yes it is a centered r squared...but I was wondering since the text just mentioned that by doing some operation, we can get (3) from (2)...I was wondering how.. $\endgroup$ – commentallez-vous Jan 5 at 17:12
  • $\begingroup$ Well, you have to understand that (2) is a covariance matrix of the vector $\beta$ whereas (3) is just an individual variance of a specific estimated coefficient $\beta_j$. You'd essentially be working with the $j$-th diagonal element of the matrix in (2) (variances). $\endgroup$ – StatsStudent Jan 5 at 17:24
  • $\begingroup$ But (2) is the inverse of the var-cov matrix X'X, so I have no idea about the properties or what the inverse of X'X is doing here. $\endgroup$ – commentallez-vous Jan 5 at 17:28
  • $\begingroup$ Do you have much knowledge in matrix algebra (I just don't have a good sense of your level here). If not, I'd highly recommend you check out chapter 5 of Applied Linear Statistical Models Fifth Edition by Kutner et al. for a good crash course. I think this will also many of your questions. If I have time later, I'll try to work this out for you and write up a complete answer. $\endgroup$ – StatsStudent Jan 5 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.