2
$\begingroup$

I have a posterior distribution over a set of parameters denoted by $\theta = \{w, \phi, \lambda\}$ and the posterior is the joint distribution given the observed data i.e. $P(\theta | D)$. Now, this posterior is a multivariate normal distribution (due to the approximate way it is estimated).

Now I want to find out the marginal distributions from this joint. So, for example, $P(\lambda)$ could be found my marginalizing over $w$ and $\phi$. Is there an easy way to do this for such MVN distributions? Also, can I find the mode for this marginal distribution easily?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Luca, if these are still the distributions you were discussing in your previous question, the independence of $w,\phi$ and $\lambda$ gives you the marginals immediately - the $q(w)$, $q(\phi)$ and $q(\lambda)$ you had in the previous question. $\endgroup$
    – Glen_b
    Commented Feb 11, 2014 at 11:37
  • $\begingroup$ Yes, they are sort of related. I have so many doubts, I am trying to figure them out one by one! $\endgroup$
    – Luca
    Commented Feb 11, 2014 at 11:58

1 Answer 1

Reset to default
2
$\begingroup$

The marginal of a multivariate normal is a univariate normal. Say your posterior distribution is

$$ p(\theta|D) = Normal(\mu, \sigma) $$

with $\mu$ a 3 dimensional vector (position 1 corresponding to $w$, position 2 corresponding to $\phi$ and position 3 corresponding to $\lambda$) and similarly for the 3x3 covariance matrix.

Then the marginal $p(\lambda|D) = Normal(\mu_3, \sigma_{33})$. You simply pick up the corresponding position from the mean vector and covariance matrices.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks Jurgen. So, for the standard deviation, I only need to look at the appropriate diagonal term? What about the covariance in the posterior distribution assuming that the model parameters are not independent? Do they not play any role once the marginalisation is performed? $\endgroup$
    – Luca
    Commented Feb 11, 2014 at 12:01
  • 1
    $\begingroup$ Correct: only the diagonal term of the covariance matrix matters for the marginal. $\endgroup$
    – Jurgen
    Commented Feb 11, 2014 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.