1
$\begingroup$

I found out that we can calculate some estimator depends on the objective function. Where if we want to minimize the least square $\sum (x_i - \theta)^2$ the best estimator is the mean. And if we want to minimize the absolute difference $\sum |x_i - \theta|$ the best estimator is the median. I was wondering, how about if I want to minimize M-estimator $\sum \rho (x_i - \theta)$, where this objective value usually called as Huber Estimator. How can I compute the estimator for this objective value? Thank you

$\endgroup$
  • $\begingroup$ It won't necessarily exist in closed form -- it depends on your choice of $\rho$. For most situations you can only minimize the loss function iteratively; for a few an explicit closed form may exist. $\endgroup$ – Glen_b -Reinstate Monica Nov 14 '16 at 16:55
  • $\begingroup$ Thanks for the response. I found that we need to compute an algorithm to get the huber estimator, but do you know some reference that I could follow easily? Because I read from a book written by Casella, that huber estimator can estimate value between mean and median, which is this is good because we can implement both properties (sensitivity to outliers data and robustness). But I don't know how to compute the Huber estimator $\endgroup$ – Jyanto Nov 16 '16 at 15:33
  • $\begingroup$ Off the top of my head I don't have an easy reference, no. It might be a good question in its own right. However, if I get some time I'll try to either outline an approach or point to something that does. $\endgroup$ – Glen_b -Reinstate Monica Nov 16 '16 at 22:12
0
$\begingroup$

A solution won't necessarily exist in closed form -- it depends on your choice of $ρ$.

For most situations you can only minimize the loss function iteratively; for a few $\rho$ functions an explicit closed form may exist.

There are a variety of suitable univariate optimization techniques. Any of the standard optimization methods can be used (e.g. the univariate optimizer optimize in R uses Brent's method), but it's fairly common to use iterative reweighted least squares starting from a good robust estimate (often, but not always the median).

In the case of the Huber $M$-estimator a well-chosen trimmed mean may provide an excellent starting point.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the response. I found the reference of Huber estimator, where the $\rho (t) = \frac{1}{2} t^2$ when $|t|< k$ and $\rho (t) = k|t| - \frac{1}{2} k^2$ when $|t| \geq k$. I read from projecteuclid.org/download/pdf_1/euclid.aoms/1177703732 . But I don't understand yet $\endgroup$ – Jyanto Nov 28 '16 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.