Estimate gaussian background noise Let's say I have 3 Random Variables $X_1, X_2, X_b$ where "$b$" stands for "background". Each one of them is Gaussian with $N(\mu_i, \sigma^2_i)$ for $ i\in\{1,2,b\}$. I will assume $\mu_b=0$. Now I make $N$ experiments which measure the variables $X_1+X_b, X_2+X_b$ (where $X_b$ is measured at the same time for both of them) and I want to estimate all the $\mu_i$'s and $\sigma_i$'s.
I know how the estimate the means by taking the average of the results (because $\mu_b = 0$). Also I can easily estimate $\sigma_j^2 + \sigma_b^2$ because if the fact that $X_j+X_b = N(\mu_j,\sigma_j^2 + \sigma_b^2)$... but I need another estimator so I can get all the values for $\sigma$'s!
I thought about using the off-diagonal elements of the co-variance matrix (which are supposed to be $\sigma_b^2$) but I get huge problem when they are negative. Can someone help me find the missing estimator?
 A: So we have $(X_{i1},X_{i2},X_{ib})$ independent random vectors where also the three components are independent, with normal distributions
with parameters $(\mu_1,\mu_2,\mu_b=0,\sigma^2_1, \sigma^2_2, \sigma^2_b)$ but you only observe $(Y=(Y_{i1},Y_{i2})^T= (X_{i1}+X_{ib}, X_{i2}+X_{ib})^T$ thus this random vector has positively correlated components, and a bivariate normal distribution (follows from the independence assumptions).  
The bivariate normal distribution of $Y$ has expectation $\mu=(\mu_1, \mu_2)^T$ and variance covariance matrix $\Sigma$ with diagonal equal to $\sigma^2_1+\sigma^2_b, \sigma^2_2+\sigma^2_b$ and off-diagonal is $\sigma^2_b$.  If your empirical covariance matrix has negative off-diagonal component, then that throws the model in doubt. 
This can be estimated straightforwardly with maximum likelihood,  let us write the loglikelihood function. You can gleen information about the multinormal distribution from https://en.wikipedia.org/wiki/Multivariate_normal_distribution
The loglikelihood function is 
$$
\ell(\mu,\Sigma) = -N \log 2 \pi - \frac{N}{2} \log \mid \Sigma \mid - \frac12 \sum_{i=1}^N (y_i - \mu_i)^T \Sigma^{-1} (y_i-\mu_i)
$$
Now you can differentiate that first with respect to $\mu$, setting equal to zero and solve, finding that $\bar{y}$ is the MLE of $\mu$. Then we can nsert that into $\ell$ above, and get the concentrated lok likelihood for $\Sigma$ as 
$$
\tilde{\ell}(\Sigma) = -\frac{N}{2}\log \mid \Sigma \mid - \frac12 \sum_{iu=1}^N  (y_i-\bar{y})^T \Sigma^{-1} (y_i-\bar{y})
$$
I would just program that (in R, for instance) and use numerical optimization to find the maximum.  To that end, it will help to parametrize $\Sigma$ in a way that guarantees positive definiteness.  Here is a paper giving ways of doing that:  https://pdfs.semanticscholar.org/2ff5/5b99d6d94d331670719bb1df1827b4d502a7.pdf
(I might come back giving more details of that!)
Another approach might be to use methods for semidefinite optimization, that is, constrained optimization with the constraint that the covariance matrices are positive semidefinite. 
