# Estimation of joint errors

Suppose we have multiple normal random variables, each which is a normal distribution with it's own mean but joint variance:

$$X_i \sim N(\mu_i, \sigma^2)$$ Now suppose we collect data from these variables, $$D=\{x_{i,j}\}$$.

What methods can one use to estimate the common parameter $$\sigma^2$$?

One can assume we have some priors $$\mu_i\sim N(0, \sigma_m^2), \sigma \sim Ga(\alpha, \beta)$$, and one could then write down the posterior by integrating over all $$\mu_i$$, but it seems there should be an easier "trick" that I'm missing.

For independent samples from two such populations with the same variance, consider using the pooled variance $$S_p^2 = \frac{(n_1 - 1)S_1^2 + (n_2-1)S_2^2}{n_1 - n_2 - 2}$$ of the pooled 2-sample t test.

For $$g$$ independent samples use the obvious generalization, which is $$S_w^2 =$$ MS(Error) in an ordinary (unbalanced) one-way ANOVA.

A confidence interval for $$\sigma^2$$ can be found using $$\frac{S_w^2}{\sigma^2} \sim \mathsf{Chisq}(\nu),$$ where $$\nu = \sum_{i=1}^g n_i - g =$$ DF(Error)}.

Because $$0.95 = P\left(L \le \frac{\nu S_w^2}{\sigma^2} \le U\right) = P\left(\frac{\nu S_w^2}{U}\le\sigma^2\le\frac{\nu S_w^2}{L}\right)$$ a 95% CI for $$\sigma^2$$ is of the form $$\left(\frac{\nu S_w^2}{U},\,\frac{\nu S_w^2}{L}\right),$$ where $$L$$ and $$U$$ cut probability $$0.025$$ from the lower and upper tails, respectively, of $$\mathsf{Chisq}(\nu).$$

Here is a numerical example in R with three groups of different sizes $$(n_1 = 10, n_2 = 8, n_3 = 15)$$ all from normal populations with $$\sigma^2 = 15^2 = 256,$$ yielding the 95% confidence interval $$(154.95,\, 433.54)$$ for $$\sigma^2$$ or, upon taking square roots of endpoints, a 95% CI $$(12.45,20.82)$$ for $$\sigma.$$

set.seed(811)
x1 = rnorm(10, 100, 15);  x2 = rnorm(8, 130, 15);  x3 = rnorm(15, 80, 15)
sse = 9*var(x1) + 7*var(x2) + 14*var(x3)
dfe = 9 + 7 + 14
mse = sse/dfe
sse/qchisq(c(.975,.025), dfe)
[1] 154.9513 433.5414
sqrt(sse/qchisq(c(.975,.025), dfe))
[1] 12.44794 20.82166

• Much simpler. Now I wonder if the gamma posterior gives the same estimate somehow :) Commented Sep 1, 2019 at 4:08
• Very likely, using a relatively non-informative prior would give a 95% Bayesian credible (posterior) interval that is numerically similar to my 95% CI based on a chi-squared distribution. (In addition, recall that chi-squared distn's are members of the gamma family.) Commented Sep 1, 2019 at 4:14