Let $S\sim B(n,\theta), l(\theta, a) = (\theta - a)^2,\delta=\bar{X} = S/n,$ and $$\delta^*(S)=\left(S+\frac{1}{2}\sqrt{n}\right)/\left(n+\sqrt{n}\right)$$ where $B(n,\theta)$ is Bernoulli distribution, $l(\theta, a)$ is loss function

How can I show that $\delta^*$ has constant risk and is Bayes for the beta, $\beta(\sqrt{n}/2, \sqrt{n}/2)$, prior. So, $\delta^*$ would be minimax. Also if we have $\theta \ne \frac{1}{2}$, would $lim_{n\rightarrow \infty}\left[R(\theta,\delta^*)/R(\theta,\delta)\right] > 1$? Otherwise, if $\theta = \frac{1}{2}$, would that make $lim_{n\rightarrow \infty}\left[R(\theta,\delta^*)/R(\theta,\delta)\right] = 1$? How do I show this?


1 Answer 1


The problem is fully processed by Wikipedia. As stated in my book,

Lemma 2.25$^0$$^1$ If $\delta_0$ is a Bayes estimator with respect to $\pi_0$ and if $R(\theta,\delta_0) \le r(\pi_0)$ for every $\theta$ in $\Theta$, $\delta_0$ is minimax and $\pi_0$ is the least favorable distribution.

Example 2.26 (Berger, 1985a) Consider $X\sim{\mathcal B}(n,\theta)$ when the probability $\theta$ is to be estimated under the quadratic loss, $$ \mathrm L (\theta,\delta) = (\delta-\theta)^2. $$ Bayes estimators are then given by posterior expectations and, when $$\theta \sim{\cal B}e \left({\sqrt{ n} \over 2}, {\sqrt{ n} \over 2} \right)$$ the posterior mean$^2$ is $$\delta^ \ast (x) = {x+ \sqrt{n}/2 \over n+ \sqrt{ n}}.$$ Moreover, this estimator has constant risk$^3$, $$R(\theta,\delta^*) = 1/4(1+\sqrt{n})^2$$ Therefore, integrating out $\theta$, $$r(\pi) = R(\theta,\delta^*)$$ and $\delta^*$ is minimax according to Lemma 2.25. Notice the difference with the MLE, $\delta_0(x) = x/n$, for the small values of $n$, and the unrealistic concentration of the prior around $0.5$ for larger values of $n$.

[The Bayesian Choice, 2007, Chapter 2]

$^0$The statement is correcting for a typo on the range of $\theta$'s for which the inequality occurs: it should be the entire parameter set and not only the support of $\pi_0$, as shown by the counter-example of a Dirac mass prior in $\theta_0$, whose (null) risk $R(\theta_0,\delta_0)$ is equal to $r(\pi_0)$

$^1$The proof is not detailed in the book for being straightforward: if $R(\theta,\delta_0) \le r(\pi_0)$ for every $\theta$ in $\Theta$, then $$r(\pi_0) = \int_\Theta R(\theta,\delta_0)\,\pi_0(\theta)\text d\theta$$ implies that $R(\theta,\delta_0) = r(\pi_0)$ almost everywhere on the support $\Omega$ of $\pi_0$. Now, if the estimator $\delta_0$ is not minimax, it means there exists an estimator $\delta_1$ such that $$\sup_\theta R(\theta,\delta_1) < \sup_\theta R(\theta,\delta_0)=r(\pi_0)$$meaning that $$\int_\Theta R(\theta,\delta_1)\,\pi_0(\theta)\text d\theta < r(\pi_0)$$ which exhibits a contradiction.

$^2$The posterior distribution is$$\theta|X=x \sim{\cal B}e \left(\frac{\sqrt{ n}}{2}+x, \frac{\sqrt{ n}}{2}+n-x \right)$$

$^3$The risk is obtained by$$R(\theta,\delta^*) = (\mathbb E_\theta[\delta^*(X)]-\theta)^2+\dfrac{\text{var}_\theta X}{(n+\sqrt{n}/2)^2}$$

  • 1
    $\begingroup$ Thank you, this is very helpful! $\endgroup$
    – John Ling
    May 5, 2019 at 1:09
  • $\begingroup$ Can you elaborate the steps where these results are drawn from? This is just stating the results with no explanations. Thanks. $\endgroup$
    – statwoman
    Apr 4, 2021 at 20:55
  • $\begingroup$ @statwoman: my whole point in this answer was to point out that the problem of the minimaxity of this estimator was fully covered in standard Bayesian textbooks. The gaps between the lines of my resolution can be filled by one line calculations. $\endgroup$
    – Xi'an
    Apr 5, 2021 at 6:15
  • $\begingroup$ Thank you @Xi'an, this made it way clearer. Could you also address the next two question asked about the limit? Thank you. $\endgroup$
    – statwoman
    Apr 6, 2021 at 16:21
  • $\begingroup$ The $R(\theta,\delta)=E((\theta-\delta)^2)=\frac{(\theta n-S)^2}{n^2}$ where $\delta=S/n$ right? How do we analyze based on this then? @Xi'an $\endgroup$
    – statwoman
    Apr 6, 2021 at 20:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.