(A cross post after finding more appropriate tags here.)

My question is on Bayesian inference of partitioned multivariate Gaussian. To make things easier, suppose there is a 2-dimensional Gaussian, $$ X_1 \sim N(\mu_1, \sigma^2_1) \\ X_2 \sim N(\mu_2, \sigma^2_2) $$ with covariance $\sigma_{1,2}$.

Suppose we know $\sigma_{i,j}$, $\sigma^2_1$ and $\sigma^2_2$; don't know $\mu_1$, $\mu_2$ but have priors for them as, $$ \mu_1 \sim N(\theta_1, \delta^2_1) \\ \mu_2 \sim N(\theta_2, \delta^2_2) $$ Now we have an observation $x_1$ for $X_1$. By Bayesian inference we get, $$ \theta'_1 | x_1 = \frac{\delta^2_1 x_1 + \sigma^2_1 \theta_1}{\delta^2_1 + \sigma^2_1} \\ \delta'^2_1 | x_1 = \frac{\delta^2_1 \sigma^2_1}{\delta^2_1 + \sigma^2_1} $$ and by partitioned Gaussian we have, $$ X_2 | X_1 \sim N \left(\mu_2 + \frac{\sigma_{1,2}}{\sigma^2_1}(x_1 - \mu_1), \sigma^2_2 - \frac{\sigma^2_{1, 2}}{\sigma^2_1} \right) $$ Finally my question is, how to update the correlated r.v using Bayesian inference, $$ p(\mu_2 | x_1) = \frac{p(x_1 | \mu_2) p(\mu_2)}{p(x_1)} $$ since I don't know how to deal with $p(x_1 | \mu_2)$. Or maybe there's other ways around to get it? Hope you get the idea of what I'm trying to do.

  • $\begingroup$ I'm wondering if $p(x_1|\mu_2) = p(x_1|x_2)p(x_2|\mu_2)$ helps. However this introduces $x_2$ which is unknown in the final result. $\endgroup$ – ShuaiYuan Mar 16 '12 at 23:13
  • $\begingroup$ Or maybe $p(x_1|\mu_2) = p(x_1|\mu_1)p(\mu_1|\mu_2)$ since $\mu_1$ and $\mu_2$ could also form a bivariate Gaussian, while assuming $Cov(\mu_1, \mu_2)$ is known? However this introduces $\mu_1$ which is still a r.v in $p(\mu_2|x_1)$. $\endgroup$ – ShuaiYuan Mar 16 '12 at 23:19
  • $\begingroup$ I'm working on an answer, but it won't be useful to you unless you're comfortable working with matrices... $\endgroup$ – Cyan Mar 17 '12 at 14:42
  • $\begingroup$ Actually, I think I'll skip the matrices -- that'll be both easier for me and (given that your question doesn't use any matrices) in all likelihood more useful to you. $\endgroup$ – Cyan Mar 18 '12 at 3:52
  • $\begingroup$ @Cyan Appreciate your help! Actually I'm dealing with multivariate Gaussian however chose to use bivariate and eliminate matrices to make it simple to represent and understand. Looking forward to your result. $\endgroup$ – ShuaiYuan Mar 18 '12 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.