2
$\begingroup$

I have been reading up on robust standard errors and had a few questions regarding how their use in logistic regression.

I have read here that heteroscedasticity is not an issue in logistic regression as there are no residuals. But here I read there are any they are used for the calculation of the sandwich estimator, and here Woolridge outlines that robust standard errors will be useful for omitted variable bias or incorrect function form only as a one term Taylor series expansion is used. I also found this which also used the sandwich estimator, but not WLS, as it appears they do in the second link, and instead use the empirical variance.

Regarding these I had a few questions

  1. I don't really know what the robust standard error is based on from this. Is it just the empirical Fisher matrix plugged into the sandwich estimator vs using the Fisher matrix on its own?

  2. Is 1 incorrect that heteroscedasticity is a problem, just in logistic regression it doesn't deal with residuals

  3. I don't understand Woolridges point on the Taylor series, when is the Taylor series used in MLE, I never learned that they are used together. Is it common to use a Taylor series to approximate the log likelihood, and when it is used, why can't higher order terms be used?

  4. Wooldridge states that robust standard errors are robust to misspecification of functional form of the independent variables and omitted variable bias, and, as opposed to the linear case, should be used only when there is strong suspicison of either; this is further supported here, and but with the recommendation of bootstrapping, are there any theoretical difference with the boostrap vs the sandwich estimator?

I feel like I'm got some large gaps in my knowledge here, so any resources such as a textbook/lecture note/lecture video series that cover these topics together would be really appreciated.

Thanks

EDIT: Added question 1.

Improve this question
$\endgroup$
2
  • $\begingroup$ A good place to start filling in your knowledge gaps is this entry in Dave Giles' blog. $\endgroup$
    – Durden
    Commented Sep 4, 2023 at 17:52
  • $\begingroup$ Thanks for this, I had a read, it still doesn't really say what heteroscedasticity is in that. I read further and found davegiles.blogspot.com/2013/05/…, which defined it in terms of the probit where setting scale=1 may be a misspecification to be resolved with defining a function for scale. I thought this is underdispersion, not heteroscedasticity, and I'm not sure how this translates to the logit, which doesn't have a scale parameter, there is a similar comment on the post which is unanswered. $\endgroup$
    – Geoff
    Commented Sep 5, 2023 at 12:08

1 Answer 1

Reset to default
2
$\begingroup$

As Friedman once said, robust standard errors provide the right estimates for the wrong quantities if the model is misspecified. This is not very appealing. And Gould has found that for binary logistic regression, sandwich estimators can be noisy, i.e, standard error estimates can have low precision.

For logistic models we tend to use sandwich covariance estimates only when there is intra-cluster correlation, i.e., use the robust cluster sandwich covariance estimator.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for this, do you have a reference for Gould? I did a quick search and couldn't find it. $\endgroup$
    – Geoff
    Commented Sep 5, 2023 at 12:10
  • $\begingroup$ I think was in the Stata Technical Bulletin now called the Stata Journal, in the 1990s or early 2000s. I hope someone can find it. Gould showed that the mean squared error of the SE estimate can be much higher with sandwich. Sandwich favors bias reduction at some cost of increased variance. $\endgroup$
    – Frank Harrell
    Commented Sep 5, 2023 at 12:22
  • $\begingroup$ Ah ok, I think Wooldridge had similar recommendations; although, Dave Giles, referenced above, seems to be the opposite, emphasising the aversion of bias, and recommends statistical tests for heteroscedasticity from here ideas.repec.org/a/eee/econom/v25y1984i3p241-262.html. Have you come across these, and, if so, would you also recommend this approach? $\endgroup$
    – Geoff
    Commented Sep 5, 2023 at 12:39
  • $\begingroup$ I don't like statistical tests for model assumptions in most cases because of the distortion of final inferences and the low power for low $N$. $\endgroup$
    – Frank Harrell
    Commented Sep 5, 2023 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.