2
$\begingroup$

I am trying to understand how to rewrite the variance of regression coefficients in terms of the variance inflation factor. To be specific If we have a regression with design matrix X, I know that we have:

$$ \widehat{\operatorname{var}}(\hat{\beta}_j) = s^2 (X^T X)^{-1} $$

and my goal is to show that:

$$ \widehat{\operatorname{var}}(\hat{\beta}_j) = \frac{s^2}{(n-1)\widehat{\operatorname{var}}(X_j)}\cdot \frac{1}{1-R_j^2} $$

where $R_j$ is the multiple r-squared for the regression between $X_j$ and the other predictors and $s^2$ is the residual standard error.

I have seen a proof on Wikipedia (https://en.wikipedia.org/wiki/Variance_inflation_factor#Definition) but would appreciate some help, either in understanding this proof or in providing a different proof or link to a reference to read.

Improve this question
$\endgroup$
3
  • $\begingroup$ Exactly! I tried tracking down the references from the Wikipedia article but never found any other proof than this one. $\endgroup$
    – Chris Henson
    Commented Feb 21, 2020 at 0:38
  • $\begingroup$ For the first equation, the variance of j-th regression coefficients should be a scalar, the j-th diagnoal of $$s^2(X^TX)^{-1}$$ As for the proof, treat $$X_j$$ as an dependent variable like $$Y$$ so that we get $$β_{∗j}$$ with the form in the wiki proof, which is the coefficient of regression of dependent variable $$X_j$$ over covariate(independent variable) $$X_{−j}$$, and then apply basic results for Residual sum of squares $\endgroup$
    – Kuo
    Commented Jul 22, 2023 at 18:31
  • $\begingroup$ For the last step multiplying $$n-1$$, we are considering unbiased sample variance. $\endgroup$
    – Kuo
    Commented Jul 22, 2023 at 18:43

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.