0
$\begingroup$

Question:

Let $X_1, · · · , X_n$ be a random sample from $Poisson(θ)$. The prior for θ is $G(α, β)$

  1. Find the Bayesian estimator (rule) of θ under the SEL(squared error loss).

  2. Find the generalized Bayesian estimator (rule) of θ under the loss $L(θ, a) = (a−θ)^2/θ$.

Solution:

My understanding:

Assume the prior distribution $θ ∼ G(α, β)$ and suppose that we observe a sample of n Poisson data. Then, we can derive the posterior distribution via Bayes theorem as:

$θ|x ∼ G(α + n \bar{x}, β + n)$

We have squared loss, $L(\theta-a)=(\theta-a)^2$. Then the Bayes rule will be $E[\theta|x]=?$

Then I couldn't go further from here. I wanted to make sense of the theory from Wikipedia. But it didn't help. I appreciate your suggestions!

$\endgroup$
2
  • $\begingroup$ Your Wikipedia link (if you repair it) says "Using the MSE as risk, the Bayes estimate of the unknown parameter is simply the mean of the posterior distribution" so for 1 you want the mean of your posterior Gamma distribution $\endgroup$
    – Henry
    Oct 4, 2021 at 20:11
  • $\begingroup$ Hey , I am self learning bayesian inference these days , and when we have prior distribution , we multiply it with given distribution of x , $ \theta $ to get their joint distribution , and after the joint distribution by integrating over $ \theta $ we get the distribution of x , can you please show the part to determine the distribution of x , I tried but something isn't getting right , , Thank you $\endgroup$
    – simran
    Oct 7, 2021 at 10:09

1 Answer 1

0
$\begingroup$

Part 1

You want to minimize the posterior expectation of the loss.

For the squared loss, you want to minimize $E\left[(\hat\theta - \theta_{posterior})^2 \right]$.

  • This is like the 2nd moment of the distribution of $\theta_{posterior}$ about the point $\hat \theta$ (the estimate).
  • It is equal to the variance of the posterior gamma distribution plus the square of the distance between the estimate $\hat\theta$ and the mean of the posterior gamma distribution. $$E\left[(\hat\theta - \theta)^2 \right] = \text{Var}(\theta) + (\hat\theta - E\left[\theta_{posterior}\right])^2$$
  • This is minimized when $\hat\theta = E\left[\theta_{posterior}\right]$
  • The mean of the posterior gamma distribution is described by $$E[\theta_{posterior}] = \frac{\alpha_{posterior}}{\beta_{posterior}}$$ See https://en.wikipedia.org/wiki/Gamma_distribution#Properties
  • We can fill in $\alpha_{posterior} = \alpha + n\bar{x}$ and $\beta_{posterior} = \beta + n$
  • So the estimator that minimises the posterior expectation of the loss is $$\hat{\theta} = \frac{\alpha + n\bar{x}}{\beta + n}$$

Part 2

For the generalized loss you want to minimize $E\left[(\hat\theta - \theta)^2 / \theta \right]$. This you can compute by considering the integration formula for expectations with gamma functions

$$\begin{array}{} E\left[ (a-x)^2/x \right] &=& \int_0^\infty (a-x)^2/x \frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x} dx \\ &=& \int_0^\infty (a-x)^2 \frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-2}e^{-\beta x} dx \\ &=& \beta \frac{\Gamma(\alpha-1)}{\Gamma(\alpha)} \int_0^\infty (a-x)^2 \frac{\beta^{\alpha-1}}{\Gamma(\alpha-1)}x^{\alpha-2}e^{-\beta x} dx \\ \end{array}$$

and so you can compute the expectation $E\left[ (a-x)^2/x \right]$ for a gamma function as the expectation $E\left[ (a-x)^2 \right]$ for another gamma function where the parameter $\alpha$ is one less (and multiply by a constant $\beta \frac{\Gamma(\alpha-1)}{\Gamma(\alpha)} = \beta/(\alpha-1)$). So like above for the squared loss, the estimate is again the same as the mean of the posterior gamma distribution, but with $\alpha$ one less (this happens to be the maximum/mode of the posterior gamma distribution).

$\endgroup$
6
  • $\begingroup$ How did you compute the estimator that minimizes the loss ? I see that Wikipedia shows the same result. en.wikipedia.org/wiki/Bayes_estimator $\endgroup$
    – ForestGump
    Oct 5, 2021 at 1:03
  • $\begingroup$ @ForestGump I just looked up the mean of a gamma distribution. Or did you mean how I know that the estimator should be the mean? $\endgroup$ Oct 5, 2021 at 5:40
  • $\begingroup$ How did u get this $\hat{\theta} = \frac{\alpha + n\bar{x}}{\beta + n}$? $\endgroup$
    – ForestGump
    Oct 5, 2021 at 17:48
  • $\begingroup$ @ForestGump I have worked out now the derivation in different steps, in the answer. $\endgroup$ Oct 6, 2021 at 16:08
  • $\begingroup$ How did u find $\alpha_{posterior} = \alpha + n\bar{x}$ and $\beta_{posterior} = \beta + n$? $\endgroup$
    – ForestGump
    Oct 6, 2021 at 21:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.