I think I must be doing something wrong here because my derived Bayes Risk is incredibly long and cumbersome...

Let $X|p \sim Bin(n,p)$ and $p \sim Beta(a,b)$.

Then $p|X \sim Beta(a+x,b+n-x)$ and the Bayes estimator of $p$ is given by

$$\delta(X) = \frac{a+x}{a+b+n}$$

My goal is to derive the Bayes risk of the Bayes estimator $\delta (X)$. My textbook seems to define (it's not actually clear to me...) Bayes risk under the squared error loss function as

$$E_{X,p} (\delta(X) - p)^2$$

which, I think, we can derive sequentially using the law of total expectation

$$E (\delta(X) - p)^2 = E[ E[(\delta(X) - p)^2|p]]$$

So my attempt to derive Bayes risk has first been to derive

$$E[(\delta(X) - p)^2|p]$$

Using the fact that $X |p \sim bin(n,p)$ and then, after deriving that, finding the total expectation by using $p \sim beta(a,b)$.

However intuitively I think I must be wrong, as the final Bayes risk I arrive at is incredibly long and tedious... and I don't think the textbook would give a problem with such a messy answer.

Is this wrong? And if not, is there any easier way to derive it in this situation?

I looked for similar answers on here for deriving Bayes risk and amazingly couldn't find any other than ddiscussions on the meaning of Bayes risk


The decomposition you propose is correct: \begin{align*} \mathbb{E} (\delta(X) - p)^2 &= \mathbb{E}[ \mathbb{E}[(\delta(X) - p)^2|p]]\\ &= \mathbb{E}[ \underbrace{\text{var}_p(\delta(X)) + (\mathbb{E}[\delta(X)]-p)^2}_\text{Pythegorean theorem}]\\ &= \mathbb{E}\left[\frac{np(1-p)}{(a+b+n)^2}+\left( \frac{a+np}{a+b+n}-p\right)^2\right]\\ &=\frac{A\mathbb{E}[p^2]+B\mathbb E[p] +C}{(a+b+n)^2} \end{align*} from which you should be able to extract a rather simple expression by recovering $A,B,C$ from the decomposition of the squares. If not, the example is processed in my book, The Bayesian Choice [Examples 4.4 and 4.5].

  • 1
    $\begingroup$ Thanks Xi'an, that's indeed what I ended up at using $MSE = Var + Bias^2$. So for the final step, the expectation of $p$ is simply taken with respect to the marginal prior distribution $p \sim Beta(a,b)$, correct? The expression I end up at is quite messy, but perhaps I can simplify it a bit $\endgroup$ – Xiaomi Nov 11 '18 at 12:31
  • 1
    $\begingroup$ Also thanks for the book recommendation. I've been after a good Bayesian book to read similar to Casella's on inference. $\endgroup$ – Xiaomi Nov 11 '18 at 12:33
  • $\begingroup$ I see now, thanks, Grouping terms before substituting in $E[p/p^2]$ makes it a lot simpler. $\endgroup$ – Xiaomi Nov 11 '18 at 14:23
  • $\begingroup$ Hello I tried to look for the example 4.4 and 4.5 in your book The Bayesian Choice, but there seem to be no example 4.4 and 4.5 in Chapter 4 of the book. $\endgroup$ – john_w Nov 20 '19 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.