Can robust standard errors be less than those from normal OLS?

Question

I'm reading about Robust Standard Error Estimators for Panel Models from the developer of plm R package (Millo, 2017: 21). But my question is not about software.

In the example I see that some heteroscedasticity robust standard errors are in fact less than those in benchmark OLS. I thought that we normaly expected robust errors to be at least as large as their OLS counterparts.

Below you see the comparison of different HC procedures, for the same model with coefficients arranged by columns.

My question concerns this situation. Is it fine in general and how to interpret this fact if, i.e. in real model, such a change in s.e. changes the significance of a coefficient?

Answer 1

Yes they can be smaller than the standard variance estimates. The reason is that even in the case of linear models, robust standard errors are biased but consistent. Their variance estimates might be (severely) biased downwards due to this small-sample bias, see Imbens & Kolesar (2016, The Review of Economics and Statistics; ungated working paper version here):

Second, and this has been pointed out in the theoretical literature before [e.g. Chesher and Jewitt, 1987], without having been appreciated in the empirical literature, problems with the standard robust EHW and LZ variances and confidence intervals can be substantial even with moderately large samples (such as 50 units / clusters) if the distribution of the regressors is skewed.

Imbens, G. W., & Kolesar, M. (2016). Robust standard errors in small samples: Some practical advice. Review of Economics and Statistics, 98(4), 701-712.

Answer 2

OLS gives every point equal weight in estimating the error of the estimates. The variance/error is estimated as

$$\widehat{\text{Var}(\beta) }= \hat{\sigma} (X^TX)^{-1} $$

A weighted least squares estimator might use an estimate like

$$\widehat{\text{Var}(\beta) }= \hat{\sigma} (X^TWX)^{-1} $$

The weights might make some points more and others less important. As a result the estimates of the standard error can improve for some coefficients.

In the example figure below an OLS estimator would assume that the variance is constant. As a result it would give the points near the origin the same weight as the points far from the origin. However, a robust regressor might give the points near the origin more weight because of the smaller variance. As a result that robust estimator will estimate a lower error for the intercept term. (And as we can see with our naked eye, the intercept is clearly around 0, but OLS will not see this because it assumes equal variance)

Answer 3

If the response is, conditionally on the covariates, Gaussian, then we know that OLS are best linear unbiased estimators by Gauss-Markov theorem; no other estimator can have lower variance than OLS.

If the Gaussian assumption is not met we have no such guarantee anymore. Indeed, since the influence function (IF) of the OLS estimator is unbounded and the variance of the OLS estimator is proportional to the IF, then the variance of the estimator is unbounded as well. In practice, this means that even a single aberrant observation can drive away the OLS estimate along with its variance.

On the other hand, robust estimators (e.g. Huber (1964) DOI: 10.1214/aoms/1177703732, Hampel et al. (1984) DOI:10.1002/9781118186435, etc.) typically have a bounded IF, thus they deliver bounded estimates with bounded variance.

The point of this is that a "bad" observation will have only a limited impact on the robust estimate and its variance, whereas it can have a large effect on an estimate obtained from an estimator with unbounded IF such as OLS.