I was recently looking at some standard error calculations using the following:
- the formula for the covariance matrix in ordinary least squares (OLS)
- the "robust" formula for the heteroskedasticity-consistent covariance matrix
- the "modified-robust" formula (also called 'White' for its discoverer) for a bias-corrected and heteroskedasticity-consistent covariance matrix
For a coefficient estimate that is about -0.05, the OLS standard error is about 0.0198. Meanwhile, the other two are in good agreement with each other and both are basically equal to 1.831. When I re-run the calculations using different libraries (SciPy.stats, scikits.statsmodels, and in R) I get excellent agreement, so I believe these estimates are correct.
This feels like a really big difference to me, relative to the value of the coefficient, but I don't really know why it feels like a big difference and don't have a good quantitative way for interpreting the difference in standard errors.
I know that if there is multicollinearity, it often means there are high standard errors. Under what conditions (if any) is it valid to go in the other direction and claim that if there is a high standard error, then there is probably multicollinearity?
In my case, the regression is a very simple two-term linear model with a constant and a coefficient on an indicator variable. But there definitely could be correlation between the constant (a base wage level) and the indicator (veteran status).
Full disclosure: this is related to a homework problem, but the homework problem just asked us to calculate the above standard errors. It didn't ask us to provide any interpretation, and I am just curious. If this had just been a data set from the wild and I had seen such differences in standard errors, it probably would make me feel nervous that I'm doing something wrong. I'm asking here just to see what a principled way to actually interpret the numbers might be.