Estimating Means of a Bivariate Normal Distribution where some parameters are known

Question

I am trying to figure out how to estimate means of a bivariate normal distribution from a sample when some of the parameters are already known.

let $$ \boldsymbol{x} = \begin{bmatrix} x\\ y\\ \end{bmatrix}, \boldsymbol{\mu} = \begin{bmatrix} \mu_x\\ \mu_y\\ \end{bmatrix}, \boldsymbol{\Sigma}=\begin{bmatrix} \sigma_x & \rho\sigma_x\sigma_y\\\ \rho\sigma_x\sigma_y & \sigma_y\\ \end{bmatrix} $$ let

$\boldsymbol{x} \sim N(\boldsymbol{\mu},\boldsymbol{\Sigma})$

I have 2 cases:

In the first case the covariance matrix ($\Sigma$) is known, but both means ($\mu_x, \mu_y$) are unknown. My intuition for this case is that the best estimator for the means are the sample averages just as they would be with an unknown covariance matrix.

In the second case in addition to the covariance matrix, one of the means ($\mu_y$) is known and all I need to estimate is $\mu_x$. Furthermore, lets assume that $\rho > 0$.

In this case the best estimator for $\mu_x$ should not be the sample average. Lets decompose each individual observation into its mean and the random component.

$$ x^i = \begin{bmatrix} x^i\\ y^i\\ \end{bmatrix} = \begin{bmatrix} \mu_x + e_x^i\\ \mu_y + e_y^i\\ \end{bmatrix} $$

Since we know $\mu_y$ we know $e_y^i$ for each observation. Since $\rho > 0$, we learn about $e_x^i$ by knowing $e_y^i$, which in turn tells us something about $\mu_x$. In particular, for $\rho > 0$ the larger $e_y^i$, the larger our estimate of $e_x^i$ should be. This in turn should lead to a lower estimate of $\mu_x$.

While I have this intuition about how knowing the mean should affect estimates I am struggeling to derive the actual estimator. Does anyone have a suggestion or a link to a textbook?

Moreover, it would be great if someone could confirm that the best estimator in case 1 is indeed the sample mean.

Answer 1

In case 1, you are correct that the mle is the sample mean.

In case 2, you are correct that it is not the sample mean.

First, assume (with loss of generality) that the variances are both 1. This can be done by dividing all $x_i$ (and $\mu_x$) by $\sigma_x$ and similarly dividing all $y_i$ and $\mu_y$ by $\sigma_y$.

Take the derivative with respect to $\mu_x$ of the log-likelihood. This will be $$\sum_{i=1}^n \frac{x_{i}-\mu_x-\rho y_{i}+\mu_y \rho}{1-\rho^2}$$

Set this equal to 0 and solve for $\mu_x$:
$$\sum_{i=1}^n (x_{i}-\rho y_{i}+\mu_y \rho)=n \mu_x$$ $$\implies \hat{\mu_x}=\frac{1}{n}\sum_{i=1}^n (x_{i}-\rho y_{i}+\mu_y \rho)=\bar{x}-\rho \bar{y}+\mu_y \rho$$

Calculate the second derivative to show it is indeed the maximum likelihood estimate.

For the original problem, notice that this is actually an estimate of the standardized mean. So, in the general case the estimate is $$\hat{\mu_x}=\bar{x}-\rho\frac{\sigma_x}{\sigma_y}\left( \bar{y}-\mu_y \right)$$