4
$\begingroup$

I was recently looking at some standard error calculations using the following:

  • the formula for the covariance matrix in ordinary least squares (OLS)
  • the "robust" formula for the heteroskedasticity-consistent covariance matrix
  • the "modified-robust" formula (also called 'White' for its discoverer) for a bias-corrected and heteroskedasticity-consistent covariance matrix

For a coefficient estimate that is about -0.05, the OLS standard error is about 0.0198. Meanwhile, the other two are in good agreement with each other and both are basically equal to 1.831. When I re-run the calculations using different libraries (SciPy.stats, scikits.statsmodels, and in R) I get excellent agreement, so I believe these estimates are correct.

This feels like a really big difference to me, relative to the value of the coefficient, but I don't really know why it feels like a big difference and don't have a good quantitative way for interpreting the difference in standard errors.

I know that if there is multicollinearity, it often means there are high standard errors. Under what conditions (if any) is it valid to go in the other direction and claim that if there is a high standard error, then there is probably multicollinearity?

In my case, the regression is a very simple two-term linear model with a constant and a coefficient on an indicator variable. But there definitely could be correlation between the constant (a base wage level) and the indicator (veteran status).

Full disclosure: this is related to a homework problem, but the homework problem just asked us to calculate the above standard errors. It didn't ask us to provide any interpretation, and I am just curious. If this had just been a data set from the wild and I had seen such differences in standard errors, it probably would make me feel nervous that I'm doing something wrong. I'm asking here just to see what a principled way to actually interpret the numbers might be.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
2
$\begingroup$

1) Are you sure you haven't misplaced a decimal point in reporting 0.02 vs. 1.8?

2) I don't see how it makes sense to describe a correlation between a constant term and a predictor variable.

3) (Most important) Are you saying you only have 1 predictor variable? In that case there is no way multicollinearity can exist, much less explain the magnitude of a standard error. Multicollinearity (often viewed through the metric of variance inflation factor or its reciprocal, tolerance) involves the extent to which one predictor can be accounted for by the other predictors being used.

Improve this answer
$\endgroup$
8
  • $\begingroup$ First, thank you for the thoughtful response. You did hit upon a confusion I was having and I wasn't aware of it until reading your answer. I have re-checked the calculations and I believe them (for whatever that's worth), so the question is: what does it mean when the heteroskedasticity-consistent standard errors are so much larger than the ordinary least squares standard errors? $\endgroup$
    – ely
    Commented Apr 14, 2012 at 20:07
  • $\begingroup$ For your second point, I'm not sure I said it clearly because your remark doesn't make sense to me. In a standard regression, say where you are estimating just an intercept and a slope on one regressor, then there are standard formulas for the covariance of those estimates. The estimated intercept term is a "constant" so to speak, but its value comes to you as a draw from some governing distribution, just like the coefficient on the lone regressor variable does, and these draws can have some underlying correlation. $\endgroup$
    – ely
    Commented Apr 14, 2012 at 20:08
  • $\begingroup$ So, to point 3, I just have the two terms, the intercept term and the slope term that is on an indicator variable. I was just wondering if multicollinearity can exist between them... that is, maybe the wage premium for veteran status is correlated with the random variable that is the sample average wage of people in veteran wage studies. It's certainly tenuous, but I was reaching for some way to understand the discrepancy between my standard error estimates. $\endgroup$
    – ely
    Commented Apr 14, 2012 at 20:10
  • $\begingroup$ EMS, there seem to be some miscommunications / misunderstandings here. rolando2's points do make sense to me. I wonder if you are working with a multi-level model of clustered data (eg, a number of measurements on each of a sample of different people) rather than a standard, 1-level regression model. Is that the case here? $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 15, 2012 at 14:35
  • $\begingroup$ I am working on some instrumental variables models, but was also asked to just do a plain-old o.l.s. regression as well, and it's just that regression that I am asking about. When you have an unknown constant as part of the regression (the intercept), this can be correlated with the other coefficients you estimate in the regression too. The correlation comes from the standard formula for the covariance matrix of the coefficients. So I do not understand the item about "correlation between a constant term and a predictor variable." $\endgroup$
    – ely
    Commented Apr 15, 2012 at 20:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.