5
$\begingroup$

There is quite some number of ways how to robustly fit a linear regression model, e.g. using M-estimation based on Tukey's biweight loss or on Huber's loss, see e.g. Wikipedia.

I got two questions about the two mentioned cases:

  1. Is it correct that these are modelling the mean (more precisely the center) of the conditional response distribution as long as the error distribution is symmetric?
  2. What location parameter are they modelling in general? Is it a specially weighted average of the response variable?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

There is quite some number of ways how to robustly fit a linear regression model, e.g. using M-estimation based on Tukey's biweight loss or on Huber's loss, see e.g. Wikipedia.

Is it correct that these are modelling the mean of the response as long as the error distribution is symmetric?

Well, estimating, but that may be what you mean by modelling -- but only if the mean exists. But they'll be reasonable estimates of the center of symmetry more generally, which will be the population median, and trimean, and pseudomedian, and any trimmed mean, and, ... and also usually the mean. For example, consider the $t$ distribution with $\nu$ degrees of freedom. If $\nu\leq 1$ there's no mean. But it still has a center of symmetry.

What location parameter are they modelling in general?

M-estimation corresponds to maximum-likelihood estimation if the loss function corresponds to $-\log(f)$ for some density $f$. This is the case for the Huber loss but not the Tukey.

Is it a specially weighted average of the response variable?

Not in general no*. But M-estimators can be obtained** by iterating a weighted average where the weights are updated at each step.

*(at least not unless 'special' is interpreted much more broadly than people generally understand the term 'weighted average' to be able to stretch)

**(in some circumstances at least)

Improve this answer
$\endgroup$
9
  • 2
    $\begingroup$ An excellent answer as usual @Glen_b . What's your idea of the best reference for this sort of thing? $\endgroup$
    – Peter Flom
    Commented Aug 29, 2014 at 10:46
  • 2
    $\begingroup$ I would say that the estimand is whatever the estimator estimates, so the question is backward in one sense. On a simple analogue: you are at liberty to regard the sample median as a robust estimator of the mean, and that may be your motivation for using it, but it is as simple or simpler to regard the sample median as an estimator of the population median. More discussion, including other points of view, at stats.stackexchange.com/questions/63386/… $\endgroup$
    – Nick Cox
    Commented Aug 29, 2014 at 11:58
  • 2
    $\begingroup$ @Peter Best kind of depends on a lot of things - what sort of treatment one seeks, for example. I also prefer to read many sources rather than one - including papers, talks, notes as well as books. But the classic references of Huber (1981) and Hampel et al (1986) are still pretty relevant (see the wikipedia page on Robust statistics for complete references and a number of other good ones). $\endgroup$
    – Glen_b
    Commented Aug 29, 2014 at 14:35
  • 2
    $\begingroup$ @Peter The book by Staudte and Sheather is a bit more recent and may be suitable for some purposes. This document by Brian Ripley isn't too bad if you want a short introduction. $\endgroup$
    – Glen_b
    Commented Aug 29, 2014 at 15:02
  • 3
    $\begingroup$ I prefer to estimate quantities that are at least somewhat easy to understand: means, mean absolute differences (for variability), quantiles, exceedance probabilities. Cumulative probability ordinal models are robust and can estimate a wide variety of quantities successfully. They do not allow "outliers" on $Y$ to affect regression coefficients, but outliers will have an impact when using these models to estimate the mean. $\endgroup$
    – Frank Harrell
    Commented Aug 29, 2014 at 22:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.