Suppose we observe $n$ independent random variables $X_1, \dots, X_n$. Suppose also that the mean and variance of each $X_i$ is unknown. If $$X_i \sim \mathrm{N}(\mu_i,\sigma^2_i)$$ then a conjugate prior for $\mu_i,\sigma^2_i$ is $$\mu_i,\sigma^2_i \sim \mathrm{N\Gamma^{-1}}(\xi_i,\lambda_i,\alpha_i,\beta_i).$$ I am interested in the case where the variance $\sigma^2$ is common to all the variables. That is $$X_i \sim \mathrm{N}(\mu_i,\sigma^2).$$ Does a joint conjugate prior exist for $\mu_1,\dots,\mu_n,\sigma^2$ and, if so, can anyone point to references which might be helpful in determining it? Any hints would be appreciated too.


To check for the existence of a conjugate prior, the likelihood must be expressed as an exponential family: $$(X_1,\ldots,X_n) \sim \sigma^{ -n} \exp \frac{-1}{2\sigma^2} \left\{ \sum_{i=1}^n (\mu_i-x_i)^2\right\}$$ Which leads to $$\mu_1,\dots,\mu_n|\sigma^2 \sim \mathcal{N}(\xi_i,\alpha\sigma^2)$$ and$$\sigma^{-2}\sim\mathcal{G}(\beta,\gamma)$$as acceptable conjugate prior, since updating to posterior means changing \begin{align*} \xi_i &\quad\text{ in } &\{\alpha^{-1}\xi+x_i\}\{\alpha^{-1}+1\}^{-1}\\ \alpha &\quad\text{ in } &\{\alpha^{-1}+1\}^{-1}\\ \beta &\quad\text{ in } &{\beta+n}\\ \gamma &\quad\text{ in } &{\gamma+\{\alpha+1\}^{-1}\sum (x_i-\xi)^2} \end{align*} or something similar.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your clear answer. When going from prior to posterior did you need to write out Bayes' theorem or is there a trick to get the updated parameters in this case? $\endgroup$ – Estacionario Sep 25 '18 at 17:49
  • 1
    $\begingroup$ That's the whole point about conjugacy, when looking at the product prior x likelihood you can identify these hyperparameters. $\endgroup$ – Xi'an Sep 25 '18 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.