Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
We know that due to multi-collinearity, the standard errors of beta estimates get inflated. But what is the mathematical basis to it?
I am looking for some mathematical relationship or something to explain this.
Like I understand if standard error of betas goes up, then t-statistics goes down and we might not be able to reject the null or the variables would appear non-significant.
But what is the mathematical relationship between multicolllinearity and inflation in variance of coefficients?
Take a look at The Analysis of Market Demand (JSTOR) by Richard Stone, Journal of the Royal Statistical Society, Vol. 108, No. 3/4 (1945), pp. 286-391. I can't find an ungated link, so here's the main result.
He gives a formula for the estimated variance of OLS regressor $\beta_k$ in a regression of $y$ on $K$ variables as $$ \frac{1}{N-K}\cdot\frac{\sigma^2_y}{\sigma^2_k}\cdot\frac{1-R^2}{1-R^2_k}, $$ where $\sigma^2_y$ is the estimated variance of $y$, $\sigma^2_k$ is the estimated variance of $x_k$, $R_k^2$ is from the regression of $x_k$ on $K-1$ remaining independent variables, and $N$ is the sample size. The set of $K$ already includes a constant.
As the independent variables get more collinear, $R^2_k$ approaches one, so the variance blows up.
