1
$\begingroup$

I have a dataset $\mathbf{D} = \{ (\tau_i, \Gamma_i) : 1 \le i \le n \}$ of observations $\tau_i = X_i + \epsilon_i$ from a $p$-dimensional Gaussian $X_i \sim \mathcal{N}(\mu, \Sigma)$ contaminated by additive noise $\epsilon_i \sim \mathcal{N}(0, \Gamma_i)$ where the $\Gamma_i$ are diagonal and observed. This gives a likelihood function of $$ \begin{aligned} L(\mu,\Sigma) &= \prod_i \int_{\mathbb{R}^p} \mathcal{N}(x_i | \tau_i, \Gamma_i)\, \mathcal{N}(x_i | \mu, \Sigma) \,dx_i \\ &= \prod_i \mathcal{N}(\tau_i | \mu, \Sigma+\Gamma_i). \end{aligned} $$

Normally I would use a Normal-inverse-Wishart prior to get a Multivariate Student's t predictive distribution, but it's not clear how to do that with this likelihood.

Is it at least possible to get a "sensible" approximation to the predictive distribution of $X$? Is there there some other prior I should consider?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

This problem is inherently difficult for Bayesian inference. Changing the prior is not going to help. You will need to resort to approximate inference techniques such as Monte Carlo, variational methods, or normal approximations.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, Tom! Actually, this problem is motivated by an application of the Expectation Propagation algorithm, but I'm stuck on constructing a decent approximate distribution on $X$. The $(\tau_i, \Gamma_i)$ are computed using EP to approximate the true likelihood function, and I'd like to then incorporate these into updating the beliefs on $X$. I see how to get a Gaussian predictive distribution on $X$ if I choose the prior to be $\mu \sim \mathcal{N}(0, \Sigma_0)$ if $\Sigma_0$ is "known", but not how to get a reasonable posterior with both $\mu$ and $\Sigma$ unknown. Thanks. $\endgroup$
    – Pine Tree
    Commented Mar 10, 2014 at 18:27
  • $\begingroup$ Like I said, you will have to do some additional approximate inference. Since you are already using EP, then you might try using it to approximate the posterior on $\mu$ and $\Sigma$ given this likelihood. But I think mean-field would be simpler and quicker. $\endgroup$
    – Tom Minka
    Commented Mar 11, 2014 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.