0
$\begingroup$

Robust estimators of scale, such as the median absolute deviation (MAD) and so on, are less affected by outliers than something like the basic standard deviation/variance. Firstly, is there a specific criterion to identify robust estimators, like a hard cutoff, or is it vague?

Secondly, would something like sigma-clipping a data set before calculating the variance/standard deviation count as a robust estimator?

By sigma-clipping, I mean iteratively:

  1. estimating the standard deviation.
  2. masking out points that are more than N standard deviations away from the mean (let's say N=4).
  3. repeating 1 & 2 on the masked dataset several times.
Improve this question
$\endgroup$
3
  • $\begingroup$ Sigma cliping only works if there is at most one outlier in your data. here is more info: stats.stackexchange.com/a/121075/603. Robustness is measured in several consistent ways. Have a look at the breakdown point en.wikipedia.org/wiki/Robust_statistics#Breakdown_point. $\endgroup$ – user603 Apr 15 '20 at 11:42
  • $\begingroup$ I was going based off the wikpedia article, but I didn't notice them give a specific criterion (like a minimum breakdown point) for an estimator to be classed as an outlier. By that definition, the sigma-clipped variance would become more robust the larger the deviance was in the outliers, and it would definitely have some level of robustness under that definition (though I'm not sure if it's possible to calculate how much in a general sense). $\endgroup$ – Marses Apr 15 '20 at 11:50
  • $\begingroup$ No. The breakdown point of the clipped variance is 0. It is the limit lim (1/n) for n -> \infty. Therefore it is not robust. It is as robust as the non clipped variance. $\endgroup$ – user603 Apr 15 '20 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.