6
$\begingroup$

Every Bayes estimator is admissible, to the best of my knowledge. (Related questions - 1,2.) I recall my professor mentioning once during a lecture that, at least as rough intuition, the converse is true as well, that is, every admissible estimator is the Bayes estimator for some choice of prior. He said something along the lines of "there are exceptions" or "regularity conditions are required".

Question: Does anyone know anything about:

  • what regularity conditions are required for the converse, every admissible estimator is the Bayes estimator for some prior, to hold?
  • and/or are there (good) counterexamples of statistical models in which (reasonable) admissible estimators are not Bayes estimators for any choice of prior?

My guess is that any counterexample might have something to do with Cromwell's rule, specifically since priors violating Cromwell's rule are well-known to artificially reduce the "effective model size". So if we had some model for which, for some reason, all priors had to violate Cromwell's rule, it would seem conceivable that there could be (reasonable) counterexamples.

As a homework problem we had to prove this converse in a very limited case: for priors not violating Cromwell's rule, and for a finite parameter space. I think the restriction to a finite parameter space was not essential though, but just to spare us from having to do convex analysis in infinite-dimensional vector spaces, since functional analysis was not listed as a prerequisite for the course. That being said, not every infinite-dimensional vector space is a Banach space for which generalizations of convex analysis apply, so conceivably we could/should still expect counterexamples to exist, but if they do exist, also expect them to have infinite parameter spaces.

EDIT: Based on this answer, another conjecture I have is that counterexamples might exist for a model where all priors have infinite Bayes risk for some reason -- maybe a Cauchy model?

$\endgroup$
  • 2
    $\begingroup$ Regarding your meta-question: if you mark it as [self-study] you'd be saying that you want hints, but you want to solve this alone; if you don't, you are saying that you want someone to answer it directly. Since this is not a homework, the [self-study] tag is not obligatory. $\endgroup$ – Tim Dec 21 '17 at 15:21
  • $\begingroup$ @Tim Oh OK -- yeah I don't care at all whether someone just gives a hint or states the entire answer -- both would be interesting to me. $\endgroup$ – Chill2Macht Dec 21 '17 at 20:24
9
$\begingroup$

Some results about Bayes and admissibility:

  1. If the Bayes risk is finite, there exists an admissible Bayes estimator, while if the Bayes risk is infinite there is no reason for the associated Bayes estimator(s) to be admissible. I cannot think of a case when all priors would have an infinite Bayes risk since the set of priors contain Dirac masses
  2. [complete class] If an estimator is admissible and the parameter set $\Theta$ is finite, then this estimator is Bayes
  3. [Blyth's theorem] If $\Theta$ is open, if the risk function $R(\theta,\delta)$ is continuous in $\theta$, and if $\delta$ is a limit of Bayes estimators in the sense that$$\lim_n\dfrac{R(\pi_n,\delta)-\min_\xi R(\pi_n,\xi)}{\pi_n(\Theta)}=0$$ then the estimator $\delta$ is admissible
  4. [Stein's theorem] If the support of the sampling density $f(\cdot|\theta)$ does not depend on $\theta$, if the loss function $L(\theta,d)$ is continuous and strictly convex in d for every $\theta$ and diverges at infinity, then every admissible estimator is a limit of Bayes estimators, corresponding to priors on a finite set
  5. The maximum likelihood estimator of the mean in the Normal mean problem, $x\sim \mathcal{N}(\theta,1)$, $\delta_0(x)=x$, is admissible under squared error loss, while not Bayes under quadratic loss but only generalised Bayes
  6. [Duanmu and Roy, 2016] For exponential families, under suitable conditions, every admissible estimator is generalized Bayes.
  7. [Farrell, 1968] "In problems of testing statistical hypotheses, there exist examples of admissible tests that cannot be generalised Bayes procedures. Although we believe the same to be true of some estimation problems, we do not have a conclusive example of an admissible estimator that is not a generalised Bayes estimator."

(All statements except 6 are available in my book, as well as Jim Berger's and Peter Hoff's.)

After digging further, I found these two exercises in Larry Brown's Fundamentals of Statistical Exponential Families:

Exercise 4.17.1, Larry Brown's Fundamentals of Statistical Exponential Families

Exercise 4.17.1, Larry Brown's Fundamentals of Statistical Exponential Families

$\endgroup$
  • 2
    $\begingroup$ This is definitely a helpful start. Do you have a reference for the third statement? We proved the first two in my course but the third is completely new to me and I would be interested to look into it more. $\endgroup$ – Chill2Macht Dec 21 '17 at 20:32
  • 2
    $\begingroup$ If I remember correctly, that comment was written when there were only three items in the list, and only the third item addressed the direction of "admissible -> Bayes". Clearly there were subsequent updates, but not having received any notifications of new answers or responses, I had no idea that such updates occurred and never thought to check for them. $\endgroup$ – Chill2Macht Apr 28 '18 at 1:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.