Is it fair to say that robust estimation should be used when the iid assumption doesn't hold? My understanding is that the sample has to be random/independent/uncorrelated but it can follow non-identical distributions - and that's when robust estimation is useful. In other words, robust estimation should be used in cases where X's and the error term are correlated but not identically distributed (could follow two normal distributions with different mean and variance). Robust estimation should produce the same consistent parameters as OLS but with different standard errors that allow for valid statistical inference.

However, if the data is clustered (panel data), then robust estimation is not necessarily the only/right tool. GLS will produce different estimates and standard errors. Is that right?

Empirical examples would be great.

  • $\begingroup$ I think that generally speaking robust is used to get estimate that are insensitive to the underlying distribution and not the i.i.d. assumption. $\endgroup$ – Michael R. Chernick Apr 13 '17 at 0:25
  • 1
    $\begingroup$ I wonder if perhaps you're asking something different than what at first glance you seem to be asking. Check wikipedia on robust statistics or the books by Huber (1981) and Hampel et al (1986) -- both called Robust Statistics. By contrast you seem to be mostly talking about robustness to errors in the variance specification /heteroskedasticity in regression-like models (per your tags -- which is an important issue, but far from the only issue most statisticians will think of when you say "robust statistics"). On the other hand ... ctd $\endgroup$ – Glen_b Apr 13 '17 at 0:26
  • $\begingroup$ ctd ... if you are taking this from the "contamination" point of view, you could be coming closer to some of the major notions in robust statistics, but those notions will apply even for iid cases. Either way you should probably clarify the intent of your question. If you're talking about robust estimation in general, the answer to your question will be qualified, since the more general sense of robust estimation could apply in considerably broader circumstances than that - it may encompass robustness to various failures of assumptions including dependence / correlation for example. ... ctd $\endgroup$ – Glen_b Apr 13 '17 at 0:30
  • $\begingroup$ ctd... and certainly even to iid cases. Consider, for example, the possibility of regression in an iid case where there's a small possibility that your errors may be very large (i.e. a distribution that looks roughly normal but has a considerably heavier tail in the far extremes). The kind of robust statistics the wikipedia article (and those books) relates to would still apply, even though the noise term in those observations may all be drawn from a single distribution. (You might call that robustness to misspecification of the distributional form perhaps) $\endgroup$ – Glen_b Apr 13 '17 at 0:42
  • $\begingroup$ Thanks a lot @Glen_b for your input. Yes, I agree. It's a bit not clear- maybe because it's not clear in my mind tbh. And yes, I am talking about robustness to errors in the variance specification/heteroskedasticity in regression models. Maybe the more appropriate description is robust estimation. At least this is how Stata folks call it. $\endgroup$ – M_M Apr 13 '17 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.