Posterior varince for multiple normal variables with identical variance

Suppose one is given $$n$$ random normal variables, all with the same variance but with different means: $$X_i\sim N(\mu_i, \sigma^2)$$

Now suppose we observe $$m_i$$ observations $$\{x_{ij}\}_{j=1}^{m_i}$$ for each variable $$X_i$$. How would one calculate the posterior distribution of $$\sigma^2$$?

I thought of using the Normal Inverse Gamma as a conjugate prior, but I don't see this working for multiple different means. I also thought of modeling this as a multidimensional distribution, but it's not clear how to enforce $$\bf \Sigma$$ to be equal to $$\sigma I$$.

• Surely a Bayesian ANOVA would do the trick? – jcken Jul 28 '20 at 20:52
• @jcken - I was looking for an explicit formula, please feel free to vet my answer below. – nbubis Jul 28 '20 at 22:26

Working out the math explicitly, and assuming an uninformative prior $$1/\sigma^4$$, I believe one should get that:
$$P(\sigma^2)\sim \left(\frac{1}{\sigma^2}\right)^{1+\sum m_i/2 -n +1 }\exp\left(\frac{1}{2\sigma^2}\sum_{i=1}^n\left[ \frac{1}{m_i}\left(\sum_{j=1}^{m_i}x_{ij}\right)^2-\sum_{j=1}^{m_i} x_{ij}^2 \right]\right)$$
This is unsurprisingly an inverse gamma distribution, $$\text{IG}(\alpha,\beta)$$, with: $$\alpha = \sum m_i/2 -n +1, \ \beta=\frac{1}{2}\sum_{i=1}^n\left[ \sum_{j=1}^{m_i} x_{ij}^2-\frac{1}{m_i}\left(\sum_{j=1}^{m_i}x_{ij}\right)^2 \right]$$ $$\longleftarrow-----\longrightarrow$$
As a sanity check, when $$n=1$$, we have that: $$\alpha=m/2,\ \beta=\frac{1}{2}\left(\sum x_j^2-m{\bar x}^2\right)=\frac{1}{2}\sum\left(x_j-\bar x\right)^2\ ,$$ exactly as we would get using a Normal-Inverse Gamma non-informative prior.