# Bayesian estimation of multivariate Gaussian from noisy observations with known error variances

I have a dataset $\mathbf{D} = \{ (\tau_i, \Gamma_i) : 1 \le i \le n \}$ of observations $\tau_i = X_i + \epsilon_i$ from a $p$-dimensional Gaussian $X_i \sim \mathcal{N}(\mu, \Sigma)$ contaminated by additive noise $\epsilon_i \sim \mathcal{N}(0, \Gamma_i)$ where the $\Gamma_i$ are diagonal and observed. This gives a likelihood function of \begin{aligned} L(\mu,\Sigma) &= \prod_i \int_{\mathbb{R}^p} \mathcal{N}(x_i | \tau_i, \Gamma_i)\, \mathcal{N}(x_i | \mu, \Sigma) \,dx_i \\ &= \prod_i \mathcal{N}(\tau_i | \mu, \Sigma+\Gamma_i). \end{aligned}

Normally I would use a Normal-inverse-Wishart prior to get a Multivariate Student's t predictive distribution, but it's not clear how to do that with this likelihood.

Is it at least possible to get a "sensible" approximation to the predictive distribution of $X$? Is there there some other prior I should consider?

• Hi, Tom! Actually, this problem is motivated by an application of the Expectation Propagation algorithm, but I'm stuck on constructing a decent approximate distribution on $X$. The $(\tau_i, \Gamma_i)$ are computed using EP to approximate the true likelihood function, and I'd like to then incorporate these into updating the beliefs on $X$. I see how to get a Gaussian predictive distribution on $X$ if I choose the prior to be $\mu \sim \mathcal{N}(0, \Sigma_0)$ if $\Sigma_0$ is "known", but not how to get a reasonable posterior with both $\mu$ and $\Sigma$ unknown. Thanks. – Pine Tree Mar 10 '14 at 18:27
• Like I said, you will have to do some additional approximate inference. Since you are already using EP, then you might try using it to approximate the posterior on $\mu$ and $\Sigma$ given this likelihood. But I think mean-field would be simpler and quicker. – Tom Minka Mar 11 '14 at 19:01