The Theil-Sen estimator is a really nifty algorithm that produces a regression line that is relatively insensitive to outliers both in the response variable and the predictor variable.

I've been wondering what a parametric "analogue" to the non-parametric Theil-Sen estimator would be. Or if there is no strict analogue, what would be a good example of a parametric model with properties similar to the Theil-Sen estimator?

With parametric analogue I mean something in the same sense as the Laplace distribution is the model that "gives you" the median. My hunch is that a bivariate Laplace distribution would have properties similar to the Theil-Sen estimator, but this is just a hunch...


As mentioned by Wilcox (2010) it does not seems like a linear regression with an error distribution that is insensitive to outliers will have similar properties as the Theil-Sen estimator. While both Theil-Sen and, for example, LAD are relative robust against outliers on the Y-axis, LAD is sensitive to outliers on the X-axis while Theil-Sen is not. For an example of this see the figure from Wilcox (2010) below:

enter image description here


Wilcox, R. R. (2010). Fundamentals of modern statistical methods: Substantially improving power and accuracy. Springer.

  • $\begingroup$ comment: "I don't believe this is correct", well, it's not a matter of faith, it's a matter of understanding what I said. I am sure you can find hundreds of papers on what I mentioned. I take my leave out of here. $\endgroup$ – user36308 Dec 17 '13 at 10:01
  • $\begingroup$ I'm sorry if I offended you. Of course it is not a matter of faith that is why I edited my question with what I believed was counterexample while still stating "please correct me if I'm wrong :)" because I'm completely open to being wrong. $\endgroup$ – Rasmus Bååth Dec 17 '13 at 10:04

I beleive, the S estimator[1] (and it's algorithm, FastS[2]) is the closest parametric equivalent to the Theil-Sen estimator.

This is because the S estimator explicitly adds a parametric assumption on the distribution of the residuals (through the tuning constant $c$) to get better efficiency at uncontaminated samples.

The FastS algorithm is implemented in the robustbase R package[3] distributed through CRAN.

There are some differences between the two approaches:

  1. FastS is more robust to outliers than Theil-Sen (the latter has a breakdown point of 0.29, the former 0.5)
  2. FastS can be computed efficiently for moderately sized dataset, including when there are more than one regressor. Theil-Sen is only defined for univariate regression.

These two differences explain why the Theil Sen estimator is essentially deprecated.

  1. Rousseeuw, P.J. and Yohai, V.J. (1984). Robust regression by means of S-estimators, In Robust and Nonlinear Time Series, J. Franke, W. Hardle and R. D. Martin (eds.). Lectures Notes in Statistics 26, 256--272, Springer Verlag, New York.
  2. Salibian-Barrera, M. Yohai, V.J. (2006). A Fast Algorithm for S-Regression Estimates. Journal of Computational and Graphical Statistics, Vol. 15, 414--427.
  3. Rousseeuw P., Croux C., Todorov V., Ruckstuhl A., Salibian-Barrera M., Verbeke T., Koller M., Maechler M. (2012). robustbase: Basic Robust Statistics. R package version 0.9--5.

One possibility consists of using flexible error distributions. This is, you have a model

$$y_j = x_j^{\top}\beta + \epsilon_j,$$

where $\epsilon_j\sim F$, and $F$ is a flexible distribution. So, for instance, in order to produce a model that is (relatively) robust to the presence of outliers and skewness, a possible choice for $F$ is a skew-t distribution (there are several types of this).

  • 1
    $\begingroup$ I don't believe this is correct (But please correct me if I'm wrong :) . See my edited question. $\endgroup$ – Rasmus Bååth Dec 17 '13 at 9:52
  • $\begingroup$ Rasmus Baath is correct that this answer is not correct. $\endgroup$ – user603 Dec 17 '13 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.