Find Bayes Estimator when Kernel of posterior is not clear

Question

Suppose $x\mid\theta \sim \operatorname{Gamma}(\frac{n}{2},2\theta)$ and $\theta \sim$ inverse Gamma$(\alpha, \beta)$

with loss function $L(\theta, d)=\frac{(\theta-d)^2}{\theta^2}$

We wish to find the Bayes estimator, ie the estimator minimizing posterior expected loss. If it was square error loss, then it would simply be the posterior mean. But since not, we must minimize the estimator with respect to the expected posterior loss.

Issues occur in that there is no obvious distribution for the posterior or way to evaluate the integrals.

The question appeared on a past exam. User Xi'an left the very interesting note that we could transfer the denominator of the loss function to our prior and then we would have the standard square error loss. In that case it would be a matter of finding the mean of the resulting posterior distribution. But I am seeing slightly different answers, as well as the answer I was given was the quotient of two non standard integrals and involved differentiating under the integral sign.

Answer 1

All of this can be solved analytically, since your prior distribution is the conjugate prior. From the specified distributions you have:

$$L_x(\theta) = \theta^{-n/2} \exp \Big( -\frac{x}{2\theta} \Big) \quad \quad \quad \pi(\theta) \propto \theta^{-\alpha-1} \exp \Big( -\frac{\beta}{\theta} \Big).$$

So the posterior is:

$$\begin{equation} \begin{aligned} \pi(\theta|x) &\propto \theta^{-n/2} \exp \Big( -\frac{x}{2\theta} \Big) \cdot \theta^{-\alpha-1} \exp \Big( -\frac{\beta}{\theta} \Big) \\[6pt] &= \theta^{-(\alpha+n/2)-1} \exp \Big( - \Big( \beta+\frac{x}{2} \Big) \frac{1}{\theta} \Big) \\[6pt] &\propto \text{Inv-Gamma} \Big( \theta \Big| \alpha+\frac{n}{2}, \beta+\frac{x}{2} \Big). \end{aligned} \end{equation}$$

So you have $\theta|x \sim\text{Inv-Gamma}(\alpha+n/2, \beta+x/2)$, which is of the same family as your prior (i.e., this is a conjugate prior). Observation of the data updates the posterior by adding to the initial hyperparameters in the prior. We have $\theta^{-1}|x \sim\text{Gamma}(\alpha+n/2, \beta+x/2)$ which means you get the risk function:

$$\begin{equation} \begin{aligned} R(d) &\equiv \mathbb{E}( L(d(x), \theta) |x) \\[6pt] &= \mathbb{E} \Big( \frac{(\theta-d(x))^2}{\theta^2} \Big| x \Big) \\[6pt] &= \mathbb{E} \Big( 1 - \frac{2d(x)}{\theta} + \frac{d(x)^2}{\theta^2} \Big| x \Big) \\[6pt] &= 1 - 2 \mathbb{E}( \theta^{-1} | x) \cdot d(x) + \mathbb{E}( \theta^{-2} |x) \cdot d(x)^2 \\[6pt] &= 1 - 2 \cdot \frac{\alpha+\tfrac{n}{2}}{\beta+\tfrac{x}{2}} \cdot d(x) + \frac{(\alpha+\tfrac{n}{2})(\alpha+\tfrac{n}{2}+1)}{(\beta+\tfrac{x}{2})^2} \cdot d(x)^2. \\[6pt] \end{aligned} \end{equation}$$

This function is a convex quadratic equation in $d$, which can easily be minimised with respect to $d$ to give you the Bayes' estimator. Minimisation of the Bayes risk gives the estimator:

$$\hat{\theta} \equiv d(x) = \frac{\beta+\tfrac{x}{2}}{\alpha+\tfrac{n}{2}+1}.$$

Answer 2

Ben's answer addresses your question completely. But I wanted to explain what happens in general. The loss function here is an example of a weighted squared error loss function. In general, a weighted squared error loss function can be written as $$L(\theta, d) = \dfrac{(\theta-d)^2}{w(\theta)}\,. $$ For example, in your situation, the weight is $w(\theta) = \theta^2$. As you mentioned, the Bayes estimator is the estimator that minimizes the expected posterior loss. That is $$d_{bayes} = \arg\min_d \int L(\theta, d) \pi(\theta|x)d\theta\,.$$

The expected posterior loss is, \begin{align*} \int L(\theta, d) \pi(\theta|x)d\theta & = \int \dfrac{(\theta - d)^2 }{w(\theta)} \dfrac{\pi(x | \theta) \pi(\theta)}{\pi(x)} d\theta\\ & = \int (\theta - d)^2 \dfrac{\pi(x | \theta)}{\pi(x)} \underbrace{\dfrac{\pi(\theta)}{w(\theta)}}_{\text{New prior kernel } \propto \pi'(\theta)}d\theta\\ & \propto \int (\theta - d)^2 \dfrac{\pi(x | \theta)}{\pi(x)} \pi'(\theta)d\theta\\ & = \int(\theta - d)^2 \pi'(\theta|x)d\theta\,. \end{align*} where $\pi'(\theta|x)$ is the new posterior distribution resulting from the new prior $\pi'(\theta)$. Then the minimizer of this new expected posterior loss, is the posterior mean with respect to $\pi'(\theta|x)$.

As probabilityislogic pointed out, the new prior may be improper, in which case the posterior may be improper, but the integral will still accurately represent the expected posterior loss.