0
$\begingroup$

Suppose we have $x=(x_1,...,x_n)|\mu,\sigma^2\sim f(x_i|\mu,\sigma^2)$ $iid$, also let $\mu\sim p(\mu)$ and $\sigma^2\sim \pi(\sigma^2)$ be prior distributions. Here $f,p,\pi$ are generic distributions. Then the posterior is $$ p(\mu,\sigma^2|x)= c\times L(\mu,\sigma^2)p(\mu)\pi(\sigma^2), $$ where $c$ is the normalizing constant and $L(\mu,\sigma^2)=f(x|\mu,\sigma^2)$ is the likelihood function. Suppose I want to estimate $\phi=\phi(\mu,\sigma^2)=\mu/\sigma^{3/2}$, what is the Bayesian estimator of $\phi$? Can I just compute $$ E(\phi|x)=\int\int \phi\times p(\mu,\sigma^2)d\mu d\sigma^2. $$ or, do I have to obtain the posterior distribution of $\phi$ first, then compute the posterior expectation of $\phi$? In which cases I can obtain Bayes' estimator of a function of the parameters directly from the joint posterior distribution of those parameters?

My doubt is due to the fact that the Bayesian estimator under quadratic loss (which is the posterior expectation) is not invariant under transformation. So, I would be glad if somebody could clarify this for me.

$\endgroup$
2
  • $\begingroup$ Could you clarify your statement that "the Bayesian estimator is not invariant under transformation"? $\endgroup$
    – frank
    Commented Mar 18, 2022 at 5:25
  • $\begingroup$ suppose $\hat{\theta}=E(\theta|x)$ is a Bayesian estimator (under quadratic loss function). If we want to estimate some function of $\theta$, say $g(\theta)$, the Bayesian estimate of $g(\theta)$ is not $g(\hat{\theta})$. Unlike, the MLE enjoys this property. $\endgroup$
    – Ludwig
    Commented Mar 18, 2022 at 10:45

1 Answer 1

1
$\begingroup$

Let's say you want to estimate the posterior on $\phi$, you can do the following; $$ \begin{aligned} p(\phi \vert x) &= \int d\mu \int d\sigma^2 \ p(\phi, \mu , \sigma^2\vert x)\\ &= \int d\mu \int d\sigma^2\ p(\mu, \sigma^2 \vert x)p(\phi \vert \mu, \sigma^2)\\ &= \int d\mu \int d\sigma^2\ c L(\mu, \sigma^2)p(\mu)p(\sigma^2)\delta\left(\sigma^2 - \frac{\mu}{\phi}^{2/3}\right)\\ &= \int d\mu\ c L\left(\mu, \sigma^2=\frac{\mu}{\phi}^{2/3}\right)p(\mu)p\left(\sigma^2 = \frac{\mu}{\phi}^{2/3}\right), \end{aligned} $$ where $\delta(y)$ is the Dirac delta and $p(\phi \vert \mu, \sigma^2)$ is a Dirac delta because $\phi$ is a function of $\mu$ and $\sigma^2$. And based on that, the expected value of $\phi$ is given by $$ \begin{aligned} E(\phi \vert x) &= \int d\phi \ \phi p(\phi \vert x)\\ &= \int d\phi \ \phi \int d\mu\ \ c L\left(\mu, \sigma^2=\frac{\mu}{\phi}^{2/3}\right)p(\mu)p\left(\sigma^2 = \frac{\mu}{\phi}^{2/3}\right)\\ &= \int d\phi \ \phi\int d\mu d\sigma^2 \ p(\mu, \sigma^2\vert x)\delta\left(\phi -\frac{\mu}{\sigma^{3/2}}\right). \end{aligned} $$

You can see that the two methods you proposed are therefore equivalent. Of course, now the prior on $\phi$ is then completely controlled by the prior on $\mu$ and $\sigma^2$.

$\endgroup$
7
  • $\begingroup$ I see, so one could obtain the estimate of $\phi$ directly. But, why people opt to obtain the posterior distribution of $\phi$, then compute the expectation? See for instance: jstor.org/stable/2683302?seq=1. The interest is a function of the parameters $\gamma=\lambda/(\lambda+\mu)$, they could had computed $E(\gamma|D)$ directly, why did they found first the distribution? $\endgroup$
    – Ludwig
    Commented Mar 18, 2022 at 11:18
  • $\begingroup$ As stated in the paper, "Since one of our main objectives is to graph this density function and hence the corresponding distribution function ...." That might explain why they want to have the entire distribution. And indeed, there is more flexibility once the posterior of the parameter of interest is known as all the statistical quantities, e.g., variance, credible interval, etc., can be easily calculated. $\endgroup$
    – Peter Pang
    Commented Mar 18, 2022 at 16:12
  • $\begingroup$ I see, so no way to obtain a credible interval for $\phi$ directly, right? $\endgroup$
    – Ludwig
    Commented Mar 18, 2022 at 16:23
  • $\begingroup$ As for the credible region, if we consider $\iint\limits_{\{\mu,\sigma^2:\phi_1<\mu/\sigma^2<\phi_2\}} p(\mu,\sigma^2|x)d\mu d\sigma^2 =1-\alpha $, is this the credible interval? $\endgroup$
    – Ludwig
    Commented Mar 18, 2022 at 16:48
  • $\begingroup$ For credible interval, it would be $\int_{\phi_{\rm min}}^{\phi_{\rm max}}d\phi\int_{\mathcal{V}}d\mu d\sigma^2p(\mu, \sigma^2 \vert x)\delta(\phi - \mu/\sigma^{3/2}) = 1 - \alpha$. So you can have it calculated directly. And because the posterior $p(\phi\vert x) = \int_{\mathcal{V}}d\mu d\sigma^2 (\mu, \sigma^2 \vert x)\delta(\phi - \mu/\sigma^{3/2})$, everything can be directly calculated from $\mu$ and $\sigma^2$. But again, just have the posterior on $\phi$ calculated would be more easy and direct, as you don't need to write whole expression for everything you do. $\endgroup$
    – Peter Pang
    Commented Mar 18, 2022 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.