Admissible Bayes Rule In the following wikipedia entry
https://en.wikipedia.org/wiki/Admissible_decision_rule
it is written that
"Bayes rules with respect to proper priors are virtually always admissible"
What do they mean by "virtually always"? I know that it needs to be unique to guarantee admissibility. Is this why they say "virtually always" but not "always"?
 A: Here are some sections from my book, The Bayesian Choice:

Proposition 2.4.22 If a prior distribution density $\pi$ is strictly positive on $\Theta$, with finite Bayes risk and the risk
  function, $R(\theta,\delta)$, is a continuous function of $\theta$ for
  every $\delta$, the Bayes estimator $\delta^\pi$ is admissible.
Proposition 2.4.23 If the Bayes estimator associated with a prior $\pi$ is unique, it is admissible.
Notice that Proposition 2.4.22 contains the assumption that
  the Bayes risk is finite. Otherwise, every estimator is, in a way, a
  Bayes estimator. On the other hand, some
  admissibility results can also be established for improper priors. This is why we
  prefer to call generalized Bayes estimators the estimators associated
  with an infinite Bayes risk, rather those corresponding to an improper
  prior. This choice implies that the Bayes estimators of different
  quantities associated with the same prior distribution can be simultaneously
  regular Bayes estimators and generalised Bayes estimators, depending on what they
  estimate. This also guarantees that regular Bayes estimators will always
  be admissible, as shown by the following result.
Proposition 2.4.25 If a Bayes estimator, $\delta^\pi$,
  associated with a (proper or improper) prior $\pi$ and a strictly
  convex loss function, is such that the Bayes
  risk, $$ r(\pi)  =  \int_{\Theta}
  R(\theta,\delta^\pi) \pi(\theta) \,d\theta, $$ is finite, $\delta^\pi$
  is admissible.
Example 2.4.28 Consider $x\sim \mathcal{N}_p(\theta,I_p)$.
  If the parameter of interest is $||\theta||^2$ and the prior distribution
  is the Lebesgue measure on $\mathbb{R}^p$, since
  $\mathbb{E}^\pi[||\theta||^2|x]  =  \mathbb{E}[||y||^2]$, with $y\sim \mathcal{N}_p(x,I_p)$, the Bayes estimator under quadratic loss is
  $$
\delta^\pi(x)  =  ||x||^2 +p.
$$
  This generalised Bayes estimator is not admissible because it is dominated by $\delta_0(x) = ||x||^2-p$. Since the classical
  risk is $$R(\theta,\delta^{\pi}) = {\mathrm{var}}(\|x\|^2)+4p^2$$ the Bayes risk is infinite.

However, if one modifies the quadratic loss into a weighted version, as in$$L(\theta,\delta)=\dfrac{(\delta-||\theta||^2)}{2||\theta||^2+1}$$ the Bayes risk is finite and the resulting Bayes estimator is admissible.
A: The Wikipedia entry needs to formalize such statement for sure so please take this answer with a grain of salt. I suspect it has to do with what you mention and the conditions under which Bayes procedures are admissible. 
If there is only one Bayes estimator for a given prior, $\delta_{\pi}$, then it must be admissible. Furthermore, if $\delta_{\pi}$ is Bayes and 
i) Unique, or
ii) support$(\pi)=\Theta$ and $\Theta$ is finite, or
iii) support$(\pi)=\Theta$ and $R(\theta,\delta)$ is continuous w.r.t. 
then $\delta_{\pi}$ is admissible. Condition iii), in spite of being restrictive, is satisfied is most practical situations nonetheless, e.g. for exponential families, or when the loss function is bounded and continuous.
I hope this helps
