6
$\begingroup$

Suppose I have a number of samples drawn from a normal distribution $x_i \sim \mathcal{N}(\mu,C)$ with $i = 1 \dots n$. I can make observations $z_i = x_i + e_i$ for those samples which are perturbed by noise $e_i$ of known characteristic $e_i \sim \mathcal{N}(0,\Sigma_i)$. Note that the distribution of the observation noise is different for each $z_i$.

I would like to estimate $\mu$ and $C$ given the $z_i$.

Since $e_i$ is unbiased $\mu$ should just be $\mathbb{E}[z_i]$. In case $\Sigma_i = D$ for all $i$, $C = \operatorname{Var}(z_i) - D$ assuming independence between the noise. Would it be safe to say that $C = \operatorname{Var}(z_i) - \mathbb{E}[\Sigma_i]$ in the general case?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ (1) What does $\mathbb{E}[\Sigma_i]$ mean when it is stipulated that $\Sigma_i$ is a "known characteristic"? Are you suggesting there is some prior distribution involved? (2) Because there is no guarantee that $\text{Var}(z_i) - \mathbb{E}[\Sigma_i]$ is positive definite, you might want to worry a bit about the potential inadmissibility or unrealism of your estimator. $\endgroup$
    – whuber ♦
    Commented Feb 19, 2013 at 21:28
  • $\begingroup$ (1) $\mathbb{E}[\Sigma_i] = 1/n \sum_{i=1..n} \Sigma_i$ (2) The same would be true for when $\Sigma_i = D$ for all $i$ as well. How do I get around this problem? And yes, I worry about the admissibility of the estimator, hence my question :) $\endgroup$
    – Jakob
    Commented Feb 19, 2013 at 21:51
  • 1
    $\begingroup$ If $\Sigma_i$ is known, and $\Sigma_i = L_i L_i'$, if you let $x_i^* = L_i^{-1} x$ (and similarly for $z$ and $e$), then depending on how you tend to look at things that may help a bit. $\endgroup$
    – Glen_b
    Commented Feb 19, 2013 at 22:06
  • 1
    $\begingroup$ Jakob, are the $x_i$ numbers or vectors? Your use of capital $\Sigma$ for the variance suggests you are considering them vectors but your edit makes sense only if they are numbers. $\endgroup$
    – whuber ♦
    Commented Feb 20, 2013 at 16:43
  • 1
    $\begingroup$ @whuber Ah sorry. Currently I am happy with $x_i$ being univariate, but I think I might have to extend it to the multivariate case later. I did an update to the notation to make it more clear, but reverted it again since then its out of line with the comments. $\endgroup$
    – Jakob
    Commented Feb 20, 2013 at 19:46

1 Answer 1

Reset to default
2
$\begingroup$

I think I've figured it out now. As stated in the comments, I think there is indeed a problem with the estimators. I found another answer on using a linear estimator for a similar problem. If I didn't make any mistakes, the estimator for $\mu$ in my case should be $$\hat{\mu} = \frac{ \sum_{i=1..n} z_i/(C+\Sigma_i)}{ \sum_{i=1..n} 1/(C+\Sigma_i)}$$ Since I don't actually know $C$, I guess I could use the estimate $\hat{C}$ based on the samples. Based on the Wikipedia Article on weighted mean and then subtracting the covariance influence from the $\Sigma_i$, I can estimate the covariance as $$\hat{C} = \left(\frac{\sum w_i}{(\sum w_i)^2 - \sum w_i^2} \sum w_i(z_i - \hat{\mu}) \right) - \frac{n}{\sum w_i}$$ where $w_i = 1/\Sigma_i$.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.