1
$\begingroup$

I wanted to ensure that I understand this correctly. So would be grateful if someone can comment on whether my reasoning is correct. To take my running example, my joint distribution is given as follows:

$$ p(w, \lambda, \phi) = p(\lambda) p(w|\lambda) P(\phi) \mathcal{L}(w, \phi) $$

where $\mathcal{L}(w, \phi)$ is the likelihood term depending only on $w$ and $\phi$. I had some help today and it seems that if I am interested in the marginal distribution, say on $\lambda$, I only need to consider the terms which have some dependence on $\lambda$.

So, we can say something like for the posterior distribution:

$$ P(\lambda|w) \propto p(\lambda) p(w|\lambda) $$

Similarly,

$$ P(\phi|w) \propto p(\phi) \mathcal{L}(w, \phi) $$

So, if we could evaluate the RHS of these equations, we have the probability density functions i.e. unto the normalisation constant for $\lambda$ and $\phi$?

Now, if my joint distribution was something where we have a new dependence where $\lambda$ now also depends on $\tau$ :

$$ p(w, \lambda, \phi, \tau) = p(\lambda) p(w|\lambda) p(\lambda|\tau) p(\phi) \mathcal{L}(w, \phi) $$

Then to evaluate the posterior on $\lambda$, I will need to condition on both $w$ and $\tau$ i.e.

$$ P(\lambda|w, \tau) \propto p(\lambda) p(w|\lambda) p(\lambda|\tau) $$

So, for a given $w$ and $\tau$, we can evaluate $P(\lambda|w, \tau)$ upto the normalisation constant?

Is the above description(s) correct?

$\endgroup$
3
  • 1
    $\begingroup$ Looks right to me. $\endgroup$
    – Glen_b
    Commented Mar 13, 2014 at 6:55
  • 1
    $\begingroup$ You can't have two distributions for $\lambda$ in the same expression, $p(\lambda)$ and $p(\lambda|\tau)$. Maybe you meant to write $p(\lambda|w,\tau)\propto p(\tau)p(\lambda|\tau)p(w|\lambda)$? Otherwise it seems correct. $\endgroup$ Commented Mar 14, 2014 at 10:36
  • $\begingroup$ I did not think of the model correctly but I sort of understood the concept that I was trying to figure out eventually :-) $\endgroup$
    – Luca
    Commented Mar 14, 2014 at 11:56

1 Answer 1

1
$\begingroup$

Just going to elaborate, in case it might be useful or another novice.

Supposing we are interested in the marginal distribution of $\phi$, we start by marginalising out $\lambda$:

$$ \begin{align} P(w, \phi) &= \int P(w, \lambda, \phi) \: d\lambda \\ &= P(\phi) \prod_{i=1}^{N}P(y_i \vert w_i, \phi) \int P(w \vert \lambda)\:d\lambda \\ &= \mathcal{K} P(\phi) \prod_{i=1}^{N}P(y_i \vert w_i, \phi) \\ \end{align} $$

where $\mathcal{K}$ is a constant representing the integral term which does not depend on $\lambda$.

Now, from definition we have:

$$ P(\lambda|w) = \frac{P(\lambda, w)}{P(w)} $$

which can be written as: $$ P(\lambda|w) \propto P(\phi) \prod_{i=1}^{N}P(y_i \vert w_i, \phi) $$

as $P(w)$ is also a constant.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.