4
$\begingroup$

Let $X_1,\ldots,X_n$ be i.i.d. Bernoulli trials with probability $\theta\in(0,1)$, and let $L:(0,1)\times[0,1]\to\mathbb{R}$ be the squared loss function, i.e. $L(\theta,a)=(\theta-a)^2$. I am trying to show that the sample mean $d(\mathbf{X})=\overline{X}$ is an admissible decision rule, i.e. there does not exist a decision rule $d':\{0,1\}^n\to\mathbb{R}$ such that $R(\theta,d')\leq R(\theta,d)$ for all $\theta$, and $R(\theta,d')<R(\theta,d)$ for some $\theta$. Here, $R(\theta,d)$ denotes the risk, defined by $$R(\theta,d)=E(L(\theta,d(\mathbf{X})))=E(\theta-d(\mathbf{X}))^2.$$

I feel like there should be a straight forward way to show this, but I am not seeing it. I have shown that given a prior distribution $\text{Beta}(\alpha,\beta)$ on $\theta$, I showed that the Bayes estimator is $d_{\alpha,\beta}(\mathbf{X})=\frac{\alpha+\sum_{i=1}^n X_i}{\alpha+\beta+n}$. I believe being a Bayes estimator in this case implies that $d_{\alpha,\beta}$ is an admissible decision rule. Then, taking the limit $\alpha,\beta\to 0$, we get $d_{\alpha,\beta}(\mathbf{X})\to\overline{X}$, so if I could claim that a limit of admissible decision rules is admissible, I would be done. But I don't think this is true in general. Any ideas?

Cross-posted from MSE

Edit: There is a similar question here, however, the answer seems to use an "improper prior" distribution on $(0,1)$. I am honestly not very familiar with improper prior distributions and I am not totally convinced of their validity. If there is a solution not relying on improper priors, I would be very interested to see it.

$\endgroup$
4
  • $\begingroup$ @SextusEmpiricus Hmmm, yea it's frustrating since it feels like there are so few ways to show an estimator is unbiased other than showing it is a Bayes estimator w.r.t. some prior distribution. $\endgroup$
    – Anon
    Commented May 5 at 22:54
  • $\begingroup$ I think you can use the improper prior argument from the other similar question. If there is any other decision rule that dominates $d(\textbf{X}) = \bar{X}$, then that integral won't be minimized by $\bar{X}$. $\endgroup$ Commented May 5 at 23:52
  • 1
    $\begingroup$ The key as to whether you can use an improper prior in this case is whether or not the posterior is proper, which it is. $\endgroup$
    – jbowman
    Commented May 6 at 0:10
  • $\begingroup$ Example 8.9 in my book, The Bayesian Choice, covers the admissibility of $\bar X_n$ in the $\mathcal B(n,\theta)$ setting. $\endgroup$
    – Xi'an
    Commented May 6 at 10:24

1 Answer 1

2
$\begingroup$

In the other question we encounter an integral like

$$r(\pi) = \mathbb{E}_\pi[r(p,d)] = \int_0^1 r(p,d)\pi(p)\text{d}p\\ $$

If the decision rule $d(x) = x/n$ minimizes this integral, then there can be no other rule that dominates it (because $r(\pi)$ for such decision rule would be lower, at least when it is better in a set of values $p$ that has non-zero measure). So then it must be admissible.

Your concern is choosing an improper $\pi(p) = 1/p(1-p)$, but for the minimization of the integral this doesn't matter. We can still write it as minimizing the posterior expected loss for each $x$.

The integral can be seen as a double integral/sum and by switching the order of integration/summation we get from minimizing the risk $r(\pi)$ to minimizing each individual posterior expected loss conditional on the observation.

$$\begin{array}{rcl} \int_0^1 r(p,d)\pi(p)\text{d}p &=& \int_0^1 \mathbb{E}_{x|p}[L(p,d(x))]\pi(p)\text{d}p \\ &=& \int_0^1 \left[ \sum_{x=0}^{n} f(x|p) L(p,d(x)) \right] \pi(p)\text{d}p \\ &=& \sum_{x=0}^{n} \rlap{\phantom{\int_0^1 \left( L(p,d(x)) \right)}\underbrace{\phantom{f(x|p) \pi(p)}}_{= f(p,x) = f(x)f(p|x)}} \left[ \int_0^1 \left( L(p,d(x)) \right) f(x|p) \pi(p)\text{d}p \right] \\ &=& \sum_{x=0}^{n} f(x) \underbrace{\left[ \int_0^1 \left( L(p,d(x)) \right) f(p|x)\text{d}p \right]}_{\text{posterior expected loss} } \\ \end{array} $$ That integral in the last line is like the posterior expected loss, and for squared loss it is minimized by the posterior mean (for every $0\leq x \leq n$), so the posterior mean also minimizes the entire sum and the Bayes risk $r(\pi)$.

$\endgroup$
8
  • $\begingroup$ Improperness matters in that the Bayes risk must remain finite. $\endgroup$
    – Xi'an
    Commented May 6 at 10:13
  • $\begingroup$ The integral must be finite for this argument to imply. Bayes estimators are not necessarily admissible when the Bayes risk is infinite. $\endgroup$
    – Xi'an
    Commented May 6 at 10:32
  • $\begingroup$ @Xi'an is it überhaupt possible to have a Bayes estimator when the Bayes risk is infinite? $\endgroup$ Commented May 6 at 10:58
  • $\begingroup$ A Bayes estimator can be defined as minimising the posterior loss for each value of the observation. $\endgroup$
    – Xi'an
    Commented May 6 at 19:56
  • $\begingroup$ @Xi'an is there an example where such definition works, yet one or more of those posterior losses are infinite? (I imagine now that cases with f(x) not integrating/summing to 1, but infinity instead, might be possible?) $\endgroup$ Commented May 7 at 6:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.