Here is a problem from a practice test. Suppose that $$X_i = \mu + \epsilon_i,\quad i=1,\ldots,n\quad \epsilon_i\sim N(0,\sigma^2_1)$$ $$Y_i = \mu + \delta_i,\quad i=1,\ldots,m\quad \delta_i\sim N(0,\sigma^2_2)$$ All $\epsilon_i$'s and $\delta_i$'s are independent. The paramters $\mu, \sigma_1^2, $ and $\sigma_2^2$ are unknown. Let $\theta=m/n$, $\rho=\sigma_2^2/\sigma_1^2$. Suppose $\rho$ is known. Show that the least squares (weighted) estimator of $\mu$ is $$ \hat{\mu} = \dfrac{\rho\bar{X} + \theta\bar{Y}}{\rho+\theta}$$


I can't figure out how to use the fact that $\rho$ is known. I tried $$\hat{\mu} = \text{argmin}\left\{\sum_{i=1}^n (X_i-\mu)^2 + \sum_{i=1}^m (Y_i-\mu)^2\right\}$$ and arrived that the weighted averaged $$ \hat{\mu} = \dfrac{n\bar{X} + m\bar{Y}}{n+m}$$ But again this does not use the fact that we know what the ratio $\sigma_2^2/\sigma_1^2$ is. Any ideas?


1 Answer 1


Consider $\mathbf{Y} = \mathbf{X}\mu + \epsilon$ where the matrices are defined as follows:

$\begin{pmatrix} X_1 \\ X_2 \\ \vdots\\ X_n \\ Y_1 \\ Y_2 \\ \vdots \\ Y_m \end{pmatrix} = \begin{pmatrix} 1 \\ 1\\ \vdots \\ 1 \end{pmatrix}\begin{pmatrix} \mu \end{pmatrix} + \begin{pmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \\ \delta_1 \\ \delta_2 \\ \vdots \\ \delta_m\end{pmatrix}$

Now we consider this as a weighted least squares problem. Let $w_i = \rho$ for $i=1,\ldots,n$ and $w_i = 1$ for $i=n+1,\ldots,n+m$. Then the weighted least squares estimate of $\mu$ is

$\hat{\mu}_{WLS} = \left(\mathbf{X}^T\mathbf{W}^{-1}\mathbf{X}\right)^{-1}\mathbf{X}^T\mathbf{W}\mathbf{Y}$ where $diag(\mathbf{W}) = (w_1,\ldots,w_{n+m})=(\rho,\ldots,\rho,1,\ldots,1)$.

Solving for $\hat{\mu}$ we get

$\begin{align*} \hat{\mu} &= \dfrac{1}{\rho n + m}\left(\rho\sum_{i=1}^n X_i + \sum_{i=1}^m Y_i\right)\\ &= \dfrac{n\rho\bar{X} + m\bar{Y}}{n\rho + m}\\ &= \dfrac{\rho\bar{X} + \tfrac{m}{n}\bar{Y}}{\rho +\tfrac{m}{n}}\\ &= \dfrac{\rho\bar{X} + \theta\bar{Y}}{\rho +\theta} \end{align*}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.