Suppose one is given $n$ random normal variables, all with the same variance but with different means: $$X_i\sim N(\mu_i, \sigma^2)$$

Now suppose we observe $m_i$ observations $\{x_{ij}\}_{j=1}^{m_i}$ for each variable $X_i$. How would one calculate the posterior distribution of $\sigma^2$?

I thought of using the Normal Inverse Gamma as a conjugate prior, but I don't see this working for multiple different means. I also thought of modeling this as a multidimensional distribution, but it's not clear how to enforce $\bf \Sigma$ to be equal to $\sigma I$.

  • $\begingroup$ Surely a Bayesian ANOVA would do the trick? $\endgroup$ – jcken Jul 28 '20 at 20:52
  • $\begingroup$ @jcken - I was looking for an explicit formula, please feel free to vet my answer below. $\endgroup$ – nbubis Jul 28 '20 at 22:26

Working out the math explicitly, and assuming an uninformative prior $1/\sigma^4$, I believe one should get that:

$$P(\sigma^2)\sim \left(\frac{1}{\sigma^2}\right)^{1+\sum m_i/2 -n +1 }\exp\left(\frac{1}{2\sigma^2}\sum_{i=1}^n\left[ \frac{1}{m_i}\left(\sum_{j=1}^{m_i}x_{ij}\right)^2-\sum_{j=1}^{m_i} x_{ij}^2 \right]\right)$$

This is unsurprisingly an inverse gamma distribution, $\text{IG}(\alpha,\beta)$, with: $$\alpha = \sum m_i/2 -n +1, \ \beta=\frac{1}{2}\sum_{i=1}^n\left[ \sum_{j=1}^{m_i} x_{ij}^2-\frac{1}{m_i}\left(\sum_{j=1}^{m_i}x_{ij}\right)^2 \right]$$ $$\longleftarrow-----\longrightarrow$$

As a sanity check, when $n=1$, we have that: $$\alpha=m/2,\ \beta=\frac{1}{2}\left(\sum x_j^2-m{\bar x}^2\right)=\frac{1}{2}\sum\left(x_j-\bar x\right)^2\ ,$$ exactly as we would get using a Normal-Inverse Gamma non-informative prior.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.