Why does the standard error for betas of predictor variables increases when they are highly correlated, indicating high levels of multicolinearity? I know that multcolinearity impacts coefficients by decreasing unique variance that can be attributed to the correlated predictor variables, but I'm still not seeing why the increase in correlation increases standard errors for those particular coefficients but not others--perhaps I'm over thinking it...Is it because it is harder to discern the impact it has on other predictor variables?
Here is the gist. Lets assume some sort of optimizer is used to estimate the parameters of a linear model, and lets assume that the error function is minimized.
If there is not a lot of multi-colinearity, then the error function has a clear minimum. The optimizer is certain where to go. See the picture below. (This does not mean that the error is low.)
Now take two variables with multi-colinearity; see the picture below. There is a big field (the white spot) where the error function does not change, because increasing one coefficient can be compensated with changing the other. There is a lot of uncertainty about the exact value of the parameter; hence the bigger error.
I took the pictures from SAS community: http://www.sascommunity.org/planet/blog/category/data-mining/