How to estimate parameters of normal distribution from two samples with different variance?

Question

Let us have two samples

$ X= (X_1,\dots X_n); \quad X_i \sim N(a,\sigma^2)$

$ Y= (Y_1,\dots Y_m); \quad Y_j \sim N(a,4\sigma^2)$.

How to find an unbiased efficient estimations for $a$ and $\sigma^2$?

Some ideas:

• We know, that $\bar X$ and $S^2_{n-1}$ are both unbiased and efficient estimators.

• We can't pool samples together, because we will obtain sample with unknown distribution.

• It is unclear, why combination of estimations from separate samples e.g. $\bar X + \bar Y $ is efficient.

Answer 1

Let us pool together samples $X$ and $Y$ to sample $Z=(X_1,\dots X_n,Y_1,\dots Y_m)=(Z_1\dots Z_k)$.

Consider linear model: $$Z=a\cdot I_k+\varepsilon,$$ where $I_k=(1,\dots 1)^T$ - $(k \times 1)$ vector and $\varepsilon$ - $(k \times 1)$ error vector.
In our case $\text{cov}(\varepsilon)= diag(\sigma^2,\dots,\sigma^2,4\sigma^2,\dots,4\sigma^2)$.

Let us scale our model by $\frac12$ for least $m$ components with help of $w=(1,\dots1,\frac14,\dots,\frac14)$ $$\tilde Z = \tilde a + \tilde \varepsilon, $$ where $\tilde Z = Z \cdot \sqrt w, \quad \tilde {I_k} = I_k \cdot \sqrt w, \quad \tilde \varepsilon= \varepsilon \cdot \sqrt w$.

This is standart linear regression model with $\text{cov}(\tilde \varepsilon) = \sigma^2I_{k\times k}$.
For this model we know unbiased estimates. Estimate for mean is:

$$\hat {\tilde a}=({\tilde I_k}^T{\tilde I_k})^{-1}{\tilde I_k ^T}\tilde Z =(\sqrt w^T I_k^T I_k \sqrt w )^{-1}\sqrt w ^T I_k^T Z \sqrt w = \\= \frac1k(\sqrt w^T \sqrt w)^{-1} \sqrt w ^T \sum Z \sqrt w = \frac1k \sum Z (\sqrt w^T \sqrt w)^{-1} (\sqrt w ^T \sqrt w) = \frac1{k} \sum Z_i$$ $$ \\{\bf \hat {\tilde a}=\frac1{k} \sum Z_i = \bar Z=\frac1{(n+m)}\left( \sum X_i + \sum Y_j \right)}$$ This estimate for mean is perfectly reasonable.

Estimate for variance is: $$\hat {\sigma}^2 = \frac {RSS}{k-1} = \frac{\Vert \tilde Z-\hat {\tilde {a}} \cdot \tilde {I_k} \Vert_2}{k-1} = \frac{\Vert Z \cdot \sqrt w- \bar Z \cdot I_k \sqrt w \Vert_2}{k-1} = \frac{1}{k-1} \Vert (Z-\bar Z I_k)\sqrt w \Vert_2 $$ $$ (Z \cdot - \bar Z \cdot I_k )\sqrt w= \left( \begin{pmatrix} X_1 \\ \vdots \\ X_n \\ Y_1 \\ \vdots\\ Y_m \end{pmatrix} - \begin{pmatrix} \bar Z\\ \vdots \\ \bar Z\\ \bar Z \\ \vdots\\ \bar Z \end{pmatrix} \right)\cdot (1,\dots,1,\frac12,\dots,\frac12)- = \begin{pmatrix} X_1-\bar Z\\ \vdots \\ X_n-\bar Z\\ \frac12 (Y_1 -\bar Z) \\ \vdots\\ \frac12 (Y_m-\bar Z) \end{pmatrix} $$ $${\bf \hat\sigma^2=\frac{1}{k-1} \left( \sum (X_i-\bar Z)^2 + \frac14 \sum (Y_j - \bar Z)^2 \right)}$$

Which is reasonable too.