What is a more robust estimator than Least Squares and Least Absolute Deviation?

Question

I am trying to fit a model using rather limited data with non-linear least squares and least absolute deviation approach. My problem is that both estimators are not very robust, and so bootstrap and jackknife both give abnormally high estimates of variance.

I am wondering what are some common and more robust estimators? Similar in spirit to least squares and least absolute deviation.

I was wondering something along the lines of:

$$L(\theta) = \sum_{i} \sqrt{|\theta_i - \theta |}$$

Since the square roots reduce the impact of outliers. But I cannot find a name of such an approach, so I guess it would be non-standard.

The closest thing I can find is the "trimmed least squares" approach. Is this commonly used? And if not, why?

Answer 1

Frank Harrell, in a post to his blog some time back, criticized the ubiquitous mistake made by using symmetric measures of uncertainty with asymmetric information. His recommendation was to substitute the Gini Mean Difference (GMD). As Yitzhaki and Schechtman note in their 2015 book The Gini Methodology: A Primer on a Statistical Methodology, all of the standard measures of uncertainty can be subsumed under the more general and robust GMD metric.

In chapter 4 of their book, "Decompositions of the GMD", they discuss a GMD-based alternative, ANOGI, analysis of Gini techniques, the equivalent of ANOVA performed with the Gini coefficient. The problem is that, while the GMD and Gini coefficient metrics are widely available, to the best of my knowledge, no off-the-shelf software currently implements the ANOGI methodology.

That said, it's not clear what you mean by limited data. It may be that, no matter how robust the metric, the resulting variability will not differ by much.