Every Bayes estimator is admissible, to the best of my knowledge. (Related questions - 1,2.) I recall my professor mentioning once during a lecture that, at least as rough intuition, the converse is true as well, that is, every admissible estimator is the Bayes estimator for some choice of prior. He said something along the lines of "there are exceptions" or "regularity conditions are required".
Question: Does anyone know anything about:
- what regularity conditions are required for the converse, every admissible estimator is the Bayes estimator for some prior, to hold?
- and/or are there (good) counterexamples of statistical models in which (reasonable) admissible estimators are not Bayes estimators for any choice of prior?
My guess is that any counterexample might have something to do with Cromwell's rule, specifically since priors violating Cromwell's rule are well-known to artificially reduce the "effective model size". So if we had some model for which, for some reason, all priors had to violate Cromwell's rule, it would seem conceivable that there could be (reasonable) counterexamples.
As a homework problem we had to prove this converse in a very limited case: for priors not violating Cromwell's rule, and for a finite parameter space. I think the restriction to a finite parameter space was not essential though, but just to spare us from having to do convex analysis in infinite-dimensional vector spaces, since functional analysis was not listed as a prerequisite for the course. That being said, not every infinite-dimensional vector space is a Banach space for which generalizations of convex analysis apply, so conceivably we could/should still expect counterexamples to exist, but if they do exist, also expect them to have infinite parameter spaces.
EDIT: Based on this answer, another conjecture I have is that counterexamples might exist for a model where all priors have infinite Bayes risk for some reason -- maybe a Cauchy model?