# Conjugate prior for the gaussian distribution

Let a set of random observations $$\{X_i\}$$, such that $$X_i \sim \mathcal{N}(\mu, \sigma^2)$$. Suppose that the mean $$\mu$$ and the variance $$\sigma^2$$ both are unknown.

What is the conjugate prior for the unknown $$\mu$$ if we want it to be always positive? For the variance, we can choose a $$Gamma(\alpha, \beta)$$ distribution. Can we choose the same for the mean?

• You would not use a Gamma distribution for the variance but either a Gamma distribution for the inverse of the variance or an inverse Gamma for the variance. Commented Jul 4, 2021 at 21:00
• You can use a truncated normal distribution for the mean Commented Jul 4, 2021 at 21:00
• Thank you @Henry for your reply. The variance is positive, so why we can't use a Gamma distribution for it ? is that for convenience of conjugacy? Commented Jul 5, 2021 at 8:05
• You can use whatever you like, but only some families of distributions will be conjugate Commented Jul 5, 2021 at 8:06
• For the mean, the idea is to use a prior distribution that allows constraining the likelihood to be positive, so for that reason I am asking if that possible. Now, I know that in order to get a closed-form solution for the posterior distribution, we should use conjugate distributions only. But what if we don’t get a close form solution but instead we get a formula that we can’t write as any other known distribution? Commented Jul 5, 2021 at 8:15

It is better to look at this in a more general way. So you have a Bayesian model with a likelihood function $$L_x(\theta)$$ and a prior $$\pi(\theta)$$, say. Then the posterior is proportional to $$L_x(\theta) \cdot \pi(\theta)$$. Now you modify the prior by incorporating the knowledge that $$\theta$$ is positive, so the new prior is $$\pi(\theta) \cdot \mathbb{1}_{\theta>0}$$, renormalized by dividing by the probability that $$\theta >0$$. Let us write this new prior as $$\pi_A(\theta)\propto \pi(\theta)\cdot \mathbb{1}_A(\theta)$$, which is simply the prior $$\pi$$ restricted to the set $$A$$.
This gives a new posterior, also restricted to the set $$A$$, which is proportional to $$\pi_A(\theta \mid x) \propto L_x(\theta)\cdot \pi(\theta)\cdot \mathbb{1}_A(\theta)$$ which simply is the old posterior restricted to the set $$A$$. So the only thing you need to do is recompute the constant of proportionality.