1
$\begingroup$

Let a set of random observations $\{X_i\}$, such that $X_i \sim \mathcal{N}(\mu, \sigma^2)$. Suppose that the mean $\mu$ and the variance $\sigma^2$ both are unknown.

What is the conjugate prior for the unknown $\mu$ if we want it to be always positive? For the variance, we can choose a $Gamma(\alpha, \beta)$ distribution. Can we choose the same for the mean?

$\endgroup$
5
  • 2
    $\begingroup$ You would not use a Gamma distribution for the variance but either a Gamma distribution for the inverse of the variance or an inverse Gamma for the variance. $\endgroup$
    – Henry
    Commented Jul 4, 2021 at 21:00
  • 1
    $\begingroup$ You can use a truncated normal distribution for the mean $\endgroup$
    – Henry
    Commented Jul 4, 2021 at 21:00
  • $\begingroup$ Thank you @Henry for your reply. The variance is positive, so why we can't use a Gamma distribution for it ? is that for convenience of conjugacy? $\endgroup$ Commented Jul 5, 2021 at 8:05
  • $\begingroup$ You can use whatever you like, but only some families of distributions will be conjugate $\endgroup$
    – Henry
    Commented Jul 5, 2021 at 8:06
  • $\begingroup$ For the mean, the idea is to use a prior distribution that allows constraining the likelihood to be positive, so for that reason I am asking if that possible. Now, I know that in order to get a closed-form solution for the posterior distribution, we should use conjugate distributions only. But what if we don’t get a close form solution but instead we get a formula that we can’t write as any other known distribution? $\endgroup$ Commented Jul 5, 2021 at 8:15

1 Answer 1

2
$\begingroup$

It is better to look at this in a more general way. So you have a Bayesian model with a likelihood function $L_x(\theta)$ and a prior $\pi(\theta)$, say. Then the posterior is proportional to $L_x(\theta) \cdot \pi(\theta)$. Now you modify the prior by incorporating the knowledge that $\theta$ is positive, so the new prior is $\pi(\theta) \cdot \mathbb{1}_{\theta>0}$, renormalized by dividing by the probability that $\theta >0$. Let us write this new prior as $\pi_A(\theta)\propto \pi(\theta)\cdot \mathbb{1}_A(\theta)$, which is simply the prior $\pi$ restricted to the set $A$.

This gives a new posterior, also restricted to the set $A$, which is proportional to $$ \pi_A(\theta \mid x) \propto L_x(\theta)\cdot \pi(\theta)\cdot \mathbb{1}_A(\theta) $$ which simply is the old posterior restricted to the set $A$. So the only thing you need to do is recompute the constant of proportionality.

So, in your case, if the posterior without restriction is a normal density, then with the restriction to the positive axis it will be a truncated normal.

If you also have a conjugate prior for the variance, then the marginal posterior for the expectation will be a t distribution, the restricted posterior will be a truncated t distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.