Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Wikipedia and the R sandwich package vignette give good information about the assumptions supporting OLS coefficient standard errors and the mathematical background of the sandwich estimators. I'm still not clear how the problem of residuals heteroscedasticity is addressed though, probably because I don't fully understand the standard OLS coefficients variance estimation in the first place.
What is the intuition behind the sandwich estimator?
For OLS, you can imagine that you're using the estimated variance of the residuals (under the assumption of independence and homoscedasticity) as an estimate for the conditional variance of the $Y_i$s. In the sandwich based estimator, you're using the observed squared residuals as a plug-in estimate of the same variance which can vary between observations.
\begin{equation} \mbox{var}\left(\hat{\beta}\right) = \left(X^TX\right)^{-1}\left(X^T\mbox{diag}\left(\mbox{var}\left(Y|X\right)\right)X\right)\left(X^TX\right)^{-1} \end{equation}
In the ordinary least squares standard error estimate for the regression coefficient estimate, the conditional variance of the outcome is treated as constant and independent, so that it can be estimated consistently.
\begin{equation} \widehat{\mbox{var}}_{OLS}\left(\hat{\beta}\right) = \left(X^TX\right)^{-1}\left(r^2X^TX\right)\left(X^TX\right)^{-1} \end{equation}
For the sandwich, we eschew consistent estimation of the conditional variance and instead use a plug-in estimate of the variance of each component using the squared residual
\begin{equation} \widehat{\mbox{var}}_{RSE}\left(\hat{\beta}\right) = \left(X^TX\right)^{-1}\left(X^T\mbox{diag}\left(r_i^2\right)X\right)\left(X^TX\right)^{-1} \end{equation}
By using the plug-in variance estimate, we get consistent estimates of the variance of $\hat{\beta}$ by the Lyapunov Central Limit Theorem.
Intuitively, these observed squared residuals will mop up any unexplained error due to heteroscedasticity that would have otherwise been unexpected under the assumption of constant variance.
