8
$\begingroup$

I'm reading about Robust Standard Error Estimators for Panel Models from the developer of plm R package (Millo, 2017: 21). But my question is not about software.

In the example I see that some heteroscedasticity robust standard errors are in fact less than those in benchmark OLS. I thought that we normaly expected robust errors to be at least as large as their OLS counterparts.

Below you see the comparison of different HC procedures, for the same model with coefficients arranged by columns.

enter image description here

My question concerns this situation. Is it fine in general and how to interpret this fact if, i.e. in real model, such a change in s.e. changes the significance of a coefficient?

$\endgroup$

3 Answers 3

4
$\begingroup$

OLS gives every point equal weight in estimating the error of the estimates. The variance/error is estimated as

$$\widehat{\text{Var}(\beta) }= \hat{\sigma} (X^TX)^{-1} $$

A weighted least squares estimator might use an estimate like

$$\widehat{\text{Var}(\beta) }= \hat{\sigma} (X^TWX)^{-1} $$

The weights might make some points more and others less important. As a result the estimates of the standard error can improve for some coefficients.

In the example figure below an OLS estimator would assume that the variance is constant. As a result it would give the points near the origin the same weight as the points far from the origin. However, a robust regressor might give the points near the origin more weight because of the smaller variance. As a result that robust estimator will estimate a lower error for the intercept term. (And as we can see with our naked eye, the intercept is clearly around 0, but OLS will not see this because it assumes equal variance)

plot

$\endgroup$
1
  • $\begingroup$ thank you for the answer. I did not get the final point. Do you mean that OLS s.e.'s report insignificant intercept, and we cannot be sure if it is truely zero; but with some W we may become more confident about 0 intercept? $\endgroup$
    – garej
    Commented Sep 19, 2022 at 4:42
3
$\begingroup$

Yes they can be smaller than the standard variance estimates. The reason is that even in the case of linear models, robust standard errors are biased but consistent. Their variance estimates might be (severely) biased downwards due to this small-sample bias, see Imbens & Kolesar (2016, The Review of Economics and Statistics; ungated working paper version here):

Second, and this has been pointed out in the theoretical literature before [e.g. Chesher and Jewitt, 1987], without having been appreciated in the empirical literature, problems with the standard robust EHW and LZ variances and confidence intervals can be substantial even with moderately large samples (such as 50 units / clusters) if the distribution of the regressors is skewed.

Imbens, G. W., & Kolesar, M. (2016). Robust standard errors in small samples: Some practical advice. Review of Economics and Statistics, 98(4), 701-712.

$\endgroup$
3
  • 2
    $\begingroup$ The second link doesn't seem to work. Generally, I find that including a full citation is most helpful because titles don't change while links can become obsolete. $\endgroup$
    – dipetkov
    Commented Oct 2, 2022 at 16:58
  • $\begingroup$ Thanks @dipetkov, I added the citation. The link works for me both on mobile as well as on my computer. $\endgroup$ Commented Oct 3, 2022 at 7:11
  • $\begingroup$ Thanks. Then we can infer you are probably accessing the link through the Princeton University network. The paper is on SSRN: NBER Working Paper No. w18478 though it might not be the final version. $\endgroup$
    – dipetkov
    Commented Oct 3, 2022 at 10:07
1
$\begingroup$

If the response is, conditionally on the covariates, Gaussian, then we know that OLS are best linear unbiased estimators by Gauss-Markov theorem; no other estimator can have lower variance than OLS.

If the Gaussian assumption is not met we have no such guarantee anymore. Indeed, since the influence function (IF) of the OLS estimator is unbounded and the variance of the OLS estimator is proportional to the IF, then the variance of the estimator is unbounded as well. In practice, this means that even a single aberrant observation can drive away the OLS estimate along with its variance.

On the other hand, robust estimators (e.g. Huber (1964) DOI: 10.1214/aoms/1177703732, Hampel et al. (1984) DOI:10.1002/9781118186435, etc.) typically have a bounded IF, thus they deliver bounded estimates with bounded variance.

The point of this is that a "bad" observation will have only a limited impact on the robust estimate and its variance, whereas it can have a large effect on an estimate obtained from an estimator with unbounded IF such as OLS.

$\endgroup$
4
  • $\begingroup$ so how to interpret this fact? Should I try to find some outliers in the data and run regressions again? I've found the same effect in my own research data. $\endgroup$
    – garej
    Commented Oct 4, 2022 at 5:04
  • $\begingroup$ The above stream of robust statistcs generally recommends let observations get donweighted by the procedure (no outlier removal). There are other streams of robust statistcs that work on methods for detecting outliers.But detecting outliers is subject to uncertainty (just like hypothesis testing). If you remove outliers, inference based on the remaining observation may be biased and has to be adjusted. If you are interested on this second stream try googling for "inference after outlier removal" or something similar. $\endgroup$
    – utobi
    Commented Oct 4, 2022 at 13:13
  • $\begingroup$ to be clear, I'm a bit confused that robust s.e. make some of important variables statistically significant, while OLS s.e. - not. I'm not sure how to deal with this. $\endgroup$
    – garej
    Commented Oct 5, 2022 at 16:01
  • 1
    $\begingroup$ @garej by the Gaussian-Markov Thm, only if we are under the true model, the robust se $\geq$ then OLS se. With observed data the inequality is not true anymore, but the point is that if the observed data have outliers, a robust procedure will "kill" them whereas OLS will be driven by them. So if you suspect your data may have outliers, forget about OLS and use robust estimates. $\endgroup$
    – utobi
    Commented Oct 6, 2022 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.