First, let $T(x)$ be an estimator of $g(\theta)$ and assume we have a square error loss function defined as $$L[g(\theta),T(x)]=[g(\theta)-T(x)]^2$$ Then the posterior expected risk of $T$ is $$\rho_T(x)=\int_{\Theta}[g(\theta)-T(x)]^2\pi(\theta|x)d\theta$$ Since we know the fact that $E(Y-b)^2$ is minimized by $b=E(Y)$, thus $$T(x)=E[g(\theta)|X=x]$$ minimizes $\rho_T(x)$.

Now we want to generalize this idea to a weighted square error loss defined as $$L^*[g(\theta),T(x)]=w(\theta)[g(\theta)-T(x)]^2$$ The solution says that the Bayes estimate now is $$T^*(x)=\frac{E[g(\theta)w(\theta)|X=x]}{E[w(\theta)|X=x]}$$ I was wondering why is this? Since the posterior expected risk is now as $$\rho_T^*(x)=\int_{\Theta}w(\theta)[g(\theta)-T(x)]^2\pi(\theta|x)d\theta=E\{w(\theta)[g(\theta)-T(x)]^2|X=x \}$$ Then why this $T^*(x)$ minimize our $\rho_T^*(x)$?


  • 2
    $\begingroup$ This is Corollary 2.5.2 (page 78) in my book, The Bayesian Choice. $\endgroup$ – Xi'an Jul 21 '18 at 14:45

Well, the simplest way to look at this phenomenon is to group the posterior density in the integrand with the weighting function (since they are both functions of $\theta$) to form a product function:

$$\rho_T^* (x) = \int_\Theta [g(\theta) - T(x)]^2 \Big( w(\theta) \pi(\theta|x) \Big) d\theta.$$

Now, this product can be considered to be the kernel of a new density (which might be an improper density for some weighting functions), which we can write as $\pi^*(\theta |x) \propto w(\theta) \pi(\theta|x)$. Via analogy to the first result we then have:

$$\begin{equation} \begin{aligned} T^*(x) &= \int_\Theta g(\theta) \pi^*(\theta|x) d\theta \\[6pt] &= \int_\Theta g(\theta) \Big( \frac{w(\theta) \pi(\theta|x)}{\int_\Theta w(\theta’) \pi(\theta’|x) d\theta’} \Big) d\theta \\[6pt] &= \frac{ \int_\Theta g(\theta) w(\theta) \pi(\theta|x) d\theta}{\int_\Theta w(\theta’) \pi(\theta’|x) d\theta’} \\[6pt] &= \frac{\mathbb{E}(g(\theta) w(\theta)|X=x)}{\mathbb{E}(w(\theta)|X=x)}. \\[6pt] \end{aligned} \end{equation}$$

  • $\begingroup$ Great answer! This line of question, I believe, leads to an interesting notion about the Bayesian approach, I recently heard/read about this here: jmanton.wordpress.com/statistics Thought this might be useful for @Nan $\endgroup$ – idnavid Jul 21 '18 at 4:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.