Bayes Minimax Estimation

Question

Let $S\sim B(n,\theta), l(\theta, a) = (\theta - a)^2,\delta=\bar{X} = S/n,$ and $$\delta^*(S)=\left(S+\frac{1}{2}\sqrt{n}\right)/\left(n+\sqrt{n}\right)$$ where $B(n,\theta)$ is Bernoulli distribution, $l(\theta, a)$ is loss function

How can I show that $\delta^*$ has constant risk and is Bayes for the beta, $\beta(\sqrt{n}/2, \sqrt{n}/2)$, prior. So, $\delta^*$ would be minimax. Also if we have $\theta \ne \frac{1}{2}$, would $lim_{n\rightarrow \infty}\left[R(\theta,\delta^*)/R(\theta,\delta)\right] > 1$? Otherwise, if $\theta = \frac{1}{2}$, would that make $lim_{n\rightarrow \infty}\left[R(\theta,\delta^*)/R(\theta,\delta)\right] = 1$? How do I show this?

Answer 1

The problem is fully processed by Wikipedia. As stated in my book,

Lemma 2.25$^0$$^1$ If $\delta_0$ is a Bayes estimator with respect to $\pi_0$ and if $R(\theta,\delta_0) \le r(\pi_0)$ for every $\theta$ in $\Theta$, $\delta_0$ is minimax and $\pi_0$ is the least favorable distribution.

Example 2.26 (Berger, 1985a) Consider $X\sim{\mathcal B}(n,\theta)$ when the probability $\theta$ is to be estimated under the quadratic loss, $$ \mathrm L (\theta,\delta) = (\delta-\theta)^2. $$ Bayes estimators are then given by posterior expectations and, when $$\theta \sim{\cal B}e \left({\sqrt{ n} \over 2}, {\sqrt{ n} \over 2} \right)$$ the posterior mean$^2$ is $$\delta^ \ast (x) = {x+ \sqrt{n}/2 \over n+ \sqrt{ n}}.$$ Moreover, this estimator has constant risk$^3$, $$R(\theta,\delta^*) = 1/4(1+\sqrt{n})^2$$ Therefore, integrating out $\theta$, $$r(\pi) = R(\theta,\delta^*)$$ and $\delta^*$ is minimax according to Lemma 2.25. Notice the difference with the MLE, $\delta_0(x) = x/n$, for the small values of $n$, and the unrealistic concentration of the prior around $0.5$ for larger values of $n$.

[The Bayesian Choice, 2007, Chapter 2]

---

$^0$The statement is correcting for a typo on the range of $\theta$'s for which the inequality occurs: it should be the entire parameter set and not only the support of $\pi_0$, as shown by the counter-example of a Dirac mass prior in $\theta_0$, whose (null) risk $R(\theta_0,\delta_0)$ is equal to $r(\pi_0)$

---

$^1$The proof is not detailed in the book for being straightforward: if $R(\theta,\delta_0) \le r(\pi_0)$ for every $\theta$ in $\Theta$, then $$r(\pi_0) = \int_\Theta R(\theta,\delta_0)\,\pi_0(\theta)\text d\theta$$ implies that $R(\theta,\delta_0) = r(\pi_0)$ almost everywhere on the support $\Omega$ of $\pi_0$. Now, if the estimator $\delta_0$ is not minimax, it means there exists an estimator $\delta_1$ such that $$\sup_\theta R(\theta,\delta_1) < \sup_\theta R(\theta,\delta_0)=r(\pi_0)$$meaning that $$\int_\Theta R(\theta,\delta_1)\,\pi_0(\theta)\text d\theta < r(\pi_0)$$ which exhibits a contradiction.

---

$^2$The posterior distribution is$$\theta|X=x \sim{\cal B}e \left(\frac{\sqrt{ n}}{2}+x, \frac{\sqrt{ n}}{2}+n-x \right)$$

---

$^3$The risk is obtained by$$R(\theta,\delta^*) = (\mathbb E_\theta[\delta^*(X)]-\theta)^2+\dfrac{\text{var}_\theta X}{(n+\sqrt{n}/2)^2}$$