Consider a non-linear least squares estimation problem, where we use a kernel (i.e. M-estimator) to reduce sensitivity to outliers,e.g.

$argmin_x \sum\limits_{k=1}^n\rho_k(||r_k(x)||_2)$,

where $r_k$ are residuals, and $\rho_k$ is a robust kernel.

There are many ways to approach this problem, for example iteratively reweighted least squares (IRLS), or kernel lifting approaches. While I have a basic understanding of the pros and cons of such methods (e.g. IRLS computes the weights at the current linearization point, which is not the best possible approximation), I am unsure about how the robust kernel itself should be chosen.

In most of the papers I've encountered in the computer vision literature, the justification for a particular robust kernel $\rho_k$ goes a little bit like this:

  1. Obviously setting $\rho_k$ to L1 or L2 norms is a bad idea
  2. We experimentaly compared every available M-estimaror and chose e.g. Geman-McClure.

From my own experience, it seems to me that the actual form of the chosen kernel $\rho_k$ doesn't really matter, as long as it is linear or quadratic for small residuals and gets nearly flat after a certain threshold (that can be set using statistics from the data), but I suspect that this is more a particularity of the data I've worked with, and not a general rule.

What is the sound way to chose a robust kernel? What are the theoretical arguments that link this to properties of the dataset (references to papers are extremely welcome)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.