Model with admissible estimator(s) that are not the Bayes estimator for any choice of prior?

Question

Every Bayes estimator is admissible, to the best of my knowledge. (Related questions - 1,2.) I recall my professor mentioning once during a lecture that, at least as rough intuition, the converse is true as well, that is, every admissible estimator is the Bayes estimator for some choice of prior. He said something along the lines of "there are exceptions" or "regularity conditions are required".

Question: Does anyone know anything about:

• what regularity conditions are required for the converse, every admissible estimator is the Bayes estimator for some prior, to hold?
• and/or are there (good) counterexamples of statistical models in which (reasonable) admissible estimators are not Bayes estimators for any choice of prior?

My guess is that any counterexample might have something to do with Cromwell's rule, specifically since priors violating Cromwell's rule are well-known to artificially reduce the "effective model size". So if we had some model for which, for some reason, all priors had to violate Cromwell's rule, it would seem conceivable that there could be (reasonable) counterexamples.

As a homework problem we had to prove this converse in a very limited case: for priors not violating Cromwell's rule, and for a finite parameter space. I think the restriction to a finite parameter space was not essential though, but just to spare us from having to do convex analysis in infinite-dimensional vector spaces, since functional analysis was not listed as a prerequisite for the course. That being said, not every infinite-dimensional vector space is a Banach space for which generalizations of convex analysis apply, so conceivably we could/should still expect counterexamples to exist, but if they do exist, also expect them to have infinite parameter spaces.

EDIT: Based on this answer, another conjecture I have is that counterexamples might exist for a model where all priors have infinite Bayes risk for some reason -- maybe a Cauchy model?

Answer 1

Some results about Bayes and admissibility:

1. If the Bayes risk is finite, there exists an admissible Bayes estimator, while if the Bayes risk is infinite there is no reason for the associated Bayes estimator(s) to be admissible. I cannot think of a case when all priors would have an infinite Bayes risk since the set of priors contain Dirac masses
2. [complete class] If an estimator is admissible and the parameter set $\Theta$ is finite, then this estimator is Bayes
3. [Blyth's theorem] If $\Theta$ is open, if the risk function $R(\theta,\delta)$ is continuous in $\theta$, and if $\delta$ is a limit of Bayes estimators in the sense that$$\lim_n\dfrac{R(\pi_n,\delta)-\min_\xi R(\pi_n,\xi)}{\pi_n(\Theta)}=0$$ then the estimator $\delta$ is admissible
4. [Stein's theorem] If the support of the sampling density $f(\cdot|\theta)$ does not depend on $\theta$, if the loss function $L(\theta,d)$ is continuous and strictly convex in d for every $\theta$ and diverges at infinity, then every admissible estimator is a limit of Bayes estimators, corresponding to priors on a finite set
5. The maximum likelihood estimator of the mean in the Normal mean problem, $x\sim \mathcal{N}(\theta,1)$, $\delta_0(x)=x$, is admissible under squared error loss, while not Bayes under quadratic loss but only generalised Bayes
6. [Duanmu and Roy, 2016] For exponential families, under suitable conditions, every admissible estimator is generalized Bayes.
7. [Farrell, 1968] "In problems of testing statistical hypotheses, there exist examples of admissible tests that cannot be generalised Bayes procedures. Although we believe the same to be true of some estimation problems, we do not have a conclusive example of an admissible estimator that is not a generalised Bayes estimator."

(All statements except 6 are available in my book, as well as Jim Berger's and Peter Hoff's.)

After digging further, I found these two exercises in Larry Brown's Fundamentals of Statistical Exponential Families: