Consider estimation of $\theta$ where $X\sim \text{Bernoulli}(\theta)$. Under squared error loss, I am asked to show that $d(X)=\frac{2X+1}{4}$ is unique Bayes with respect to any prior for which $\operatorname E[\theta]=\frac12$ and $\operatorname{Var}[\theta]=\frac18$. I also have to argue that $d$ is unique minimax.

I could show that $d(X)$ is the unique Bayes estimator (the posterior mean) by taking a $\text{Beta}(\frac12,\frac12)$ prior (also discussed here for example). Indeed, $\operatorname E[\theta]=\frac12$ and $\operatorname{Var}[\theta]=\frac18$ hold for this choice of prior. But how do I show that this is the case for any prior with this mean and variance?

Since $d$ is Bayes with constant risk, this would also mean that $d$ is minimax. But can I say $d$ is unique minimax because it is unique Bayes?

  • 2
    $\begingroup$ are there missing words? is unique Bayes [missing words] with respect e.g., is the unique Bayesian estimator of $\theta$ with respect $\endgroup$ – innisfree Mar 8 at 7:27
  • $\begingroup$ No missing words. It is mentioned at the start that we are estimating $\theta$. $\endgroup$ – StubbornAtom Mar 8 at 9:46
  • 1
    $\begingroup$ @StubbornAtom "Bayes" is usually not a noun except when it means Thomas Bayes himself $\endgroup$ – Henry Mar 8 at 11:24
  • $\begingroup$ @Henry Did I claim otherwise? $\endgroup$ – StubbornAtom Mar 8 at 11:32

The posterior expectation writes as \begin{align*}\mathbb E^\pi[\theta|x]&=\dfrac{\mathbb E^\pi[\theta^2x+\theta(1-\theta)(1-x)]}{\mathbb E^\pi[\theta x+(1-\theta)(1-x)]}\\ &=\dfrac{\mathbb E^\pi[2\theta^2 x-\theta x +\theta-\theta^2]}{\mathbb E^\pi[2\theta x+1-\theta-x]}\\ &=\dfrac{\mathbb 2\frac{3}{8} x-\frac{1}{2} x +\frac{1}{2}-\frac{3}{8}}{2\frac{1}{2} x+1-\frac{1}{2}-x} \end{align*} which shows that it only depends on the first two moments of the prior.

Since this estimator is

  1. a proper Bayes estimator (i.e., with a finite risk)
  2. with a constant risk,

it is

  1. admissible (since proper Bayes),
  2. minimax (because of the constant risk) and therefore
  3. unique minimax.

Indeed, there can be no other minimax procedure since the later would dominate the constant risk minimax estimator, in contradiction with the admissible nature of this estimator. If there were another minimax estimator with a constant risk, admissibility would become jeopardised since a convex combination of the two would dominate.$^1$ But this does not prevent this estimator from being the Bayes optimal procedure associated with an infinity of priors.

$^1$A fascinating example where two different constant risk minimax estimators (co)exist is the Normal $\mathcal N_p(\mu,\mathsf I_p)$ mean estimation. Both $\delta_0(x)=x$ and $\delta_{2p-2}(x)=(1-(2p-4)||x||^{-2})x$ have a constant risk equal to $p$ and they are both dominated by $\delta_{p-2}(x)=(1-(p-2)||x||^{-2})x$, as shown by Charles Stein.

  • $\begingroup$ How do you get that expression? Am I right in saying $$E[\theta\mid X=x]=\int\theta \pi(\theta\mid x)\,d\theta=\frac{\int\theta f(x\mid \theta)\pi(\theta)\,d\theta}{\int f(x\mid \theta)\pi(\theta)\,d\theta}=\frac{E_{\pi}[\theta^{1+x}(1-\theta)^{1-x}]}{E_{\pi}\left[\theta^x(1-\theta)^{1-x}\right]}\,?$$ $\endgroup$ – StubbornAtom Mar 8 at 10:10
  • $\begingroup$ Thanks, I see that now since $x$ is binary. I guess it must be written in your form to be able to use the information on $E[\theta]$ and $\operatorname{Var}[\theta]$. The admissibility follows from uniqueness of Bayes rule, right? But which result guarantees an admissible minimax rule to be unique in this setting? $\endgroup$ – StubbornAtom Mar 8 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.