# Sample mean of Bernoulli trials is admissible under squared loss

Let $$X_1,\ldots,X_n$$ be i.i.d. Bernoulli trials with probability $$\theta\in(0,1)$$, and let $$L:(0,1)\times[0,1]\to\mathbb{R}$$ be the squared loss function, i.e. $$L(\theta,a)=(\theta-a)^2$$. I am trying to show that the sample mean $$d(\mathbf{X})=\overline{X}$$ is an admissible decision rule, i.e. there does not exist a decision rule $$d':\{0,1\}^n\to\mathbb{R}$$ such that $$R(\theta,d')\leq R(\theta,d)$$ for all $$\theta$$, and $$R(\theta,d') for some $$\theta$$. Here, $$R(\theta,d)$$ denotes the risk, defined by $$R(\theta,d)=E(L(\theta,d(\mathbf{X})))=E(\theta-d(\mathbf{X}))^2.$$

I feel like there should be a straight forward way to show this, but I am not seeing it. I have shown that given a prior distribution $$\text{Beta}(\alpha,\beta)$$ on $$\theta$$, I showed that the Bayes estimator is $$d_{\alpha,\beta}(\mathbf{X})=\frac{\alpha+\sum_{i=1}^n X_i}{\alpha+\beta+n}$$. I believe being a Bayes estimator in this case implies that $$d_{\alpha,\beta}$$ is an admissible decision rule. Then, taking the limit $$\alpha,\beta\to 0$$, we get $$d_{\alpha,\beta}(\mathbf{X})\to\overline{X}$$, so if I could claim that a limit of admissible decision rules is admissible, I would be done. But I don't think this is true in general. Any ideas?

Cross-posted from MSE

Edit: There is a similar question here, however, the answer seems to use an "improper prior" distribution on $$(0,1)$$. I am honestly not very familiar with improper prior distributions and I am not totally convinced of their validity. If there is a solution not relying on improper priors, I would be very interested to see it.

• @SextusEmpiricus Hmmm, yea it's frustrating since it feels like there are so few ways to show an estimator is unbiased other than showing it is a Bayes estimator w.r.t. some prior distribution.
– Anon
Commented May 5 at 22:54
• I think you can use the improper prior argument from the other similar question. If there is any other decision rule that dominates $d(\textbf{X}) = \bar{X}$, then that integral won't be minimized by $\bar{X}$. Commented May 5 at 23:52
• The key as to whether you can use an improper prior in this case is whether or not the posterior is proper, which it is. Commented May 6 at 0:10
• Example 8.9 in my book, The Bayesian Choice, covers the admissibility of $\bar X_n$ in the $\mathcal B(n,\theta)$ setting. Commented May 6 at 10:24

In the other question we encounter an integral like

$$r(\pi) = \mathbb{E}_\pi[r(p,d)] = \int_0^1 r(p,d)\pi(p)\text{d}p\\$$

If the decision rule $$d(x) = x/n$$ minimizes this integral, then there can be no other rule that dominates it (because $$r(\pi)$$ for such decision rule would be lower, at least when it is better in a set of values $$p$$ that has non-zero measure). So then it must be admissible.

Your concern is choosing an improper $$\pi(p) = 1/p(1-p)$$, but for the minimization of the integral this doesn't matter. We can still write it as minimizing the posterior expected loss for each $$x$$.

The integral can be seen as a double integral/sum and by switching the order of integration/summation we get from minimizing the risk $$r(\pi)$$ to minimizing each individual posterior expected loss conditional on the observation.

$$\begin{array}{rcl} \int_0^1 r(p,d)\pi(p)\text{d}p &=& \int_0^1 \mathbb{E}_{x|p}[L(p,d(x))]\pi(p)\text{d}p \\ &=& \int_0^1 \left[ \sum_{x=0}^{n} f(x|p) L(p,d(x)) \right] \pi(p)\text{d}p \\ &=& \sum_{x=0}^{n} \rlap{\phantom{\int_0^1 \left( L(p,d(x)) \right)}\underbrace{\phantom{f(x|p) \pi(p)}}_{= f(p,x) = f(x)f(p|x)}} \left[ \int_0^1 \left( L(p,d(x)) \right) f(x|p) \pi(p)\text{d}p \right] \\ &=& \sum_{x=0}^{n} f(x) \underbrace{\left[ \int_0^1 \left( L(p,d(x)) \right) f(p|x)\text{d}p \right]}_{\text{posterior expected loss} } \\ \end{array}$$ That integral in the last line is like the posterior expected loss, and for squared loss it is minimized by the posterior mean (for every $$0\leq x \leq n$$), so the posterior mean also minimizes the entire sum and the Bayes risk $$r(\pi)$$.

• Improperness matters in that the Bayes risk must remain finite. Commented May 6 at 10:13
• The integral must be finite for this argument to imply. Bayes estimators are not necessarily admissible when the Bayes risk is infinite. Commented May 6 at 10:32
• @Xi'an is it überhaupt possible to have a Bayes estimator when the Bayes risk is infinite? Commented May 6 at 10:58
• A Bayes estimator can be defined as minimising the posterior loss for each value of the observation. Commented May 6 at 19:56
• @Xi'an is there an example where such definition works, yet one or more of those posterior losses are infinite? (I imagine now that cases with f(x) not integrating/summing to 1, but infinity instead, might be possible?) Commented May 7 at 6:38