2
$\begingroup$

From what I read the Theil-Sen estimator seems to be a really cool estimator; robust and easy to understand. Would it be possible to device (or does it already exists) a Bayesian "version" of the Theil-Sen estimator so that you provide a prior for the slope and get a probability density back instead of just a point estimate? How would such an estimator look/work?

$\endgroup$
2
$\begingroup$

According to the Complete Class Theorem proved by Abraham Wald in his book "Statistical Decision Functions", an estimator is admissible if and only if it is Bayes for some prior. Hence, if the Theil-Sen estimator can't be obtained as a Bayes decision, it is inadimissible.

$\endgroup$
  • $\begingroup$ Does this mean that there is no Bayesian version of Theil-Sen? Sorry I don't really understand the answer... $\endgroup$ – Rasmus Bååth Dec 10 '12 at 7:47
  • 2
    $\begingroup$ The thought is that if it's decision-theoretically 'any good' i.e. 'admissible' in the technical sense used there, then it has formulation that is Bayesian. (which doesn't, of course, imply that it is admissible, that applying decision theoretic criteria make sense anyway, or that it's otherwise sensible). $\endgroup$ – conjugateprior Dec 10 '12 at 8:41
  • $\begingroup$ Well, then my question must have been vague. Even if I asked if the could be a Bayesian "version" I was also interested in how that "version" would look, not only if it existed... Clarified my question above... $\endgroup$ – Rasmus Bååth Dec 10 '12 at 9:24
1
$\begingroup$

Since the Theil-Sen thingy is robust to outliers by virtue of dealing with medians, I guess the natural Bayesian regression model is one with a t-distribution with unknown degrees of freedom (or other fat tailed distribution, e.g. double exponential might be closest in spirit) replacing the normal distribution of classical regression.

In this case the prior for the slope is not so important because fatter tails for Y allow occasional outliers without affecting the slope estimate. The degrees of freedom in the relevant t is, in a complete analysis, marginalised out of the joint posterior.

Some discussion of this class of models can be found in chapter 14 and 17 of Gelman's Bayesian Data Analysis. A choice of distributions is explored in chapter 8 (stacks) of the BUGS Examples Volume one.

$\endgroup$
  • $\begingroup$ I was thinking about that, but then I thouht that it seemed more like a Bayesian version of Least Absolute Deviation Regression which is robust outliers in y (if y ~ x) but not outliers in x. A nice property of the Theil-Sen is that it is both robust to outliers in x and y. $\endgroup$ – Rasmus Bååth Dec 10 '12 at 9:20
  • 1
    $\begingroup$ I don't think so. Sen seems to think that it is biased and inconsistent in the present of measurement error, as one might expect. See e.g. sentence 2 of The Theil–Sen estimator in a measurement error perspective where the 'discounted parameter' that is median-unbiasedly estimated is an attenuated version of the actual slope of interest. $\endgroup$ – conjugateprior Dec 10 '12 at 9:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.