3
$\begingroup$

Consider $x\sim N(\theta,1)$ and $\theta\sim N(0,n)$. Show that the Bayes risk is equal to $\frac{n}{n+1}$.

I know that $$r(\theta,\delta)=\int_\chi\int_\Theta L(\theta,\delta(x))\pi(\theta|x)d\theta m(x)dx$$

If $x|\mu\sim N(\mu,\sigma^2)$ and $\mu\sim N(\mu_0,\sigma_0^2)$ then $$\mu|x\sim N(\frac{\sigma^2_0}{\sigma^2+\sigma_0^2}x+\frac{\sigma^2}{\sigma^2+\sigma^2_0}\mu_0,(\frac{1}{\sigma^2}+\frac{1}{\sigma_0^2})^{-1})$$

in this case $$\pi(\theta|x)\sim N(\frac{n}{1+n}x,(\frac{n+1}{n})^{-1})$$

Following the idea of @X'ian give me here Risk and posterior expectation Bayesian Statistics

$$R(\theta,\delta)=\mathbb{E}_\theta[L(\theta,\delta(x)]=\mathbb{E}_\theta[(\theta-\delta(x))^2]$$ $$R(\theta,\delta)=\mathbb{E}_\theta[(\theta-\mathbb{E}_\theta[\delta(x)])^2]+\mathbb{E}_\theta[(\mathbb{E}_\theta[\delta(x)]-\delta(x))^2]$$

$$\mathbb{E}_\theta[(\theta-\mathbb{E}_\theta[\delta(x)])^2]=\mathbb{E}_\theta[(\theta-\mathbb{E}_\theta[\frac{n}{1+n}x])^2]=\mathbb{E}[(\frac{\theta(1+n)-n\mathbb{E}(x)}{1+n})^2]$$ $$=\mathbb{E}_\theta[(\frac{\theta(1+n)-n\theta}{1+n})^2]=\mathbb{E}_\theta[(\frac{\theta}{1+n})^2]=0$$

$$\mathbb{E}_\theta[(\mathbb{E}_\theta[\delta(x)]-\delta(x))^2]=Var(\delta(x))=Var(\frac{n}{1+n}x)=(\frac{n}{1+n})^2*1=(\frac{n}{1+n})^2=\frac{n}{1+n}$$

Is it right?

$\endgroup$
  • 1
    $\begingroup$ What is the source of the question? If nothing else is provided I would assume $L$ is quadratic loss and $\delta$ is the posterior mean. $\endgroup$ – Christoph Hanck Mar 23 '16 at 13:04
  • $\begingroup$ @ChristophHanck I take it from "The Bayesian Choice" exercise 2.30 $\endgroup$ – user72621 Mar 23 '16 at 15:06
  • $\begingroup$ Yes, it does not seem to be very explicit (@Xi'an ?), but you may indeed continue to work with quadratic loss and, thus, the posterior mean. $\endgroup$ – Christoph Hanck Mar 23 '16 at 15:51
4
$\begingroup$

(I am assuming the loss function here is the squared error loss).

You have the formula for Bayes risk right, but you have the second moment of the prior distribution wrong, and then algebra at the end wrong. I am going to proceed with the solution by first finding $R(\theta, \delta)$, and then $r(\delta, \pi)$, using the following two equations.

$$R(\theta, \delta) = E_{X|\theta} \left[(\theta - \delta(x))^2 \right] = Var_{X|\theta}(\delta(x)) + \left(\theta - E_{X|\theta}(\delta(x)) \right)^2. $$

$$r(\delta, \pi) = E_{\theta} \left[R(\theta, \delta) \right]. $$

Here $E_{X|\theta}$ is expectation with respect to the likelihood, $E_{\theta}$ is expectation with respect to the prior.

$$\delta(x) = \dfrac{n}{n+1} x. $$

\begin{align*} R(\theta, \delta) & = Var_{X|\theta}(\delta(x)) + \left(\theta - E_{X|\theta}(\delta(x)) \right)^2\\ & = Var_{X|\theta} \left(\dfrac{n}{n+1} x \right) + \left( \theta - \dfrac{n}{n+1} E_{x|\theta}[x] \right)^2\\ & = \left(\dfrac{n}{n + 1} \right)^2 + \left(\dfrac{\theta}{n+1} \right)^2 \end{align*}

Next, I find the expectation of $R(\theta, \delta)$ with respect to the prior to find Bayes risk. Remember that the second moment of the prior distribution is $n$, since $E_{\theta}[\theta^2] = Var_{\theta}[\theta] + (E_{\theta}[\theta])^2 = n$.

\begin{align*} r(\delta, \pi) & = \left(\dfrac{n}{n + 1} \right)^2 + \dfrac{E_{\theta} [\theta^2]}{(n+1)^2}\\ & = \dfrac{n^2 + n}{(n+1)^2}\\ & = \dfrac{n}{n+1}. \end{align*}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy