1
$\begingroup$

I would very much appreciate some help regarding how to interpret different robust measures of scale (Inter-quartile range or IQR, biweight midvariance, and median absolute deviation or MAD). Thus, comments or pointers to relevant documents will be really welcomed.

I am aware that none of these statistics can probably be considered "better" than the others but I would like to know the possible pros and cons of each of these statistics (or when the use of each of them is more/ less recommendable). For example, I assume that IQR may be less sensitive to effects occurring at central locations of a group's distribution (and, therefore, less interesting in cases we suspect dispersion is not uniform), but I do not know if biweight midvariances and mad differ in this (or any other relevant) regard.

Thanks in advance for any possible help!

$\endgroup$
4
  • $\begingroup$ Please search our site for breakdown point. $\endgroup$
    – whuber
    May 1, 2023 at 17:13
  • $\begingroup$ Hi whuber, thanks for your reply. I have searched in the posts with the breakdown descriptor but I could not find a direct comparison of the dispersion statistics of my question. $\endgroup$
    – user222456
    May 1, 2023 at 18:44
  • 1
    $\begingroup$ That's because you're not going to find a question specifically about arbitrary collections of statistics--certainly not those in this question. What is it, then, you are trying to find out? $\endgroup$
    – whuber
    May 1, 2023 at 19:21
  • $\begingroup$ Thanks again for your new reply. My general aim is to understand which different aspects of scale/ dispersion are grabbing these indices. A more specific question is whether or not biweight midvariances are more/ less sensitive to variation at the tails of a distribution than mad. I am asking this because I tested the 3 statistics in 2-group comparisons and obtained different results (with IQR and mad being more similar between them) and I would like to understand these results (and whether or not they relate to capturing dispersion at central/ distal locations) $\endgroup$
    – user222456
    May 2, 2023 at 15:09

1 Answer 1

1
$\begingroup$

In general, there are three types of nonparametric scale estimators: Quantile differences (Range/Interquartile range), Bickel-Lehmann dispersion (median absolute deviation, standard deviation, average absolute deviation, any estimators defined using a location estimator to estimate the deviations of all points relative to the location estimate of the sample can be seen in this class), and Bickel-Lehmann spread (besides the estimators proposed in Bickel and Lehmann's landmark paper, the popular Rousseeuw–Croux estimator and Gini's mean difference also can be seen in this class since they are a modification of the pairwise differences).

Estimators belong to the class, Bickel-Lehmann spread, are the optimal approach in terms of variance, but their computational cost is often much higher than others.

This is because the variance is mainly dependent on the sample size/pseudo sample size and the standard deviation of the underlying distribution, this principle is governed by the Central Limit Theorem (although CLT is about the sample mean, the principle can be extended to other estimators, asymptotic normality).

Suppose the sample size is n, the median absolute deviation is essentially estimating the median of all n absolute deviations, so, the pseudosample size, i.e., the acutual size related to the median is also n. The Bickel-Lehmann spread is the location of all n^2 pairwise differences, so the pseudosample size is n^2. The quantile differences (Range/Interquartile range) is just computing two quantiles, so the pseudosample size is 2.

Another factor is the distribution of pseudosample. If just considering the sample size, you might conclude that the quantile differences is much much worse than Bickel-Lehmann dispersion and spread, which is indeed not the case, they are just slightly worse. This is because, the distribution of pseudosample is a high kurtosis, monotonic decreasing distribution for pairwise differences, as proven by Hodges and Lehmann in 1954 and further in my paper, Robust estimations for semiparametric models: Moments (Robust estimations for semiparametric models: Moments (https://zenodo.org/records/8127703)(https://www.researchgate.net/publication/377974264_Robust_estimations_from_distribution_structures_Central_Moments). As we know, the variance of a location estimator is linearly depending on the standard deviation of the underlying distribution, which is the advantage of quantile differences. However, the linear relation can be dominated by the square relation, so Bickel-Lehmann spread is still better even considering this factor.

Bickel, P.J., Lehmann, E.L. (2012). Descriptive Statistics for Nonparametric Models IV. Spread. In: Rojo, J. (eds) Selected Works of E. L. Lehmann. Selected Works in Probability and Statistics. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-1412-4_45

Hodges, J. L., & Lehmann, E. L. (1954). Matching in paired comparisons. The Annals of Mathematical Statistics, 25(4), 787-791.

$\endgroup$
4
  • 3
    $\begingroup$ This is a bit spare by our standards. While the exposition about Gauss is useful, the answer contains no information to explain why the Bickel-Lehmann spread is optimal in terms of variance. Instead, readers are directed to your YouTube channel. This is now how this site works: answers should not be a platform for self-promotion, and they should contain relevant explanations. Citing works is fine & encouraged, but answers should summarize the key points. $\endgroup$
    – Sycorax
    Feb 11 at 14:20
  • $\begingroup$ Thank you for your comment, I briefly explain why the Bickel-Lehmann spread is optimal. This is because of the principle related to the central limit theorem. $\endgroup$
    – Tuobang Li
    Feb 11 at 15:24
  • $\begingroup$ Is it your intention to promote your own publications and YouTube channel in every answer that you write here? $\endgroup$
    – Sycorax
    Feb 11 at 16:16
  • 1
    $\begingroup$ I removed the YouTube channel and added new explanation. The shape of the pseudosample distribution, or kernel distribution, is also important in explaining the factors that impact the variance. $\endgroup$
    – Tuobang Li
    Feb 11 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.