2
$\begingroup$

I have a joint distribution which factorizes as follows:

$$ P(y, w, \beta) = P(y|\beta, w) P(w) P(\beta) $$

Now, I want to write the conditional distribution for $P(w|y, \beta)$, so this should be written as:

$$ P(w|y, \beta) = \frac{P(y, w, \beta)}{P(y, \beta)} = \frac{P(y|\beta, w) P(w) P(\beta)}{P(y|\beta) P(\beta)} \propto P(y|\beta, w) P(w) $$

I am not sure of the last step but I am guessing the denominator does not depend on $w$, we can treat it like a constant. My question is whether I can find the maxima for $P(w|y, \beta)$ by finding the maxima of $P(y|\beta, w) P(w)$ i.e. treat the $P(y|\beta)$ denominator as a constant. I am trying to do this in the context of figuring out some variational bayes updates where I need to compute some expectations and need the conditional distributions to do that.

$\endgroup$
3
  • 2
    $\begingroup$ Correct: $p(w\mid y,\beta)\propto p(w,y,\beta)\propto p(y\mid\beta,w)p(w)$, which is a mnemonic for "discard" things that do not depend on $w$. Normalize at the end, if necessary. $\endgroup$
    – Zen
    Feb 25, 2015 at 22:11
  • $\begingroup$ Luca you don't seem to actually ask question here, just make statements. What exactly do you need? $\endgroup$
    – Glen_b
    Mar 6, 2015 at 1:53
  • $\begingroup$ I was wondering if I could find the maxima of $P(w|y, \beta)$ by optimising just $P(y|\beta, w)P(w)$. Sorry I was not clear with that. $\endgroup$
    – Luca
    Mar 6, 2015 at 7:32

1 Answer 1

0
$\begingroup$

I was wondering if I could find the maxima of $P(w|y, \beta)$ by optimizing just $P(y|\beta, w) P(w)$.

You can because you are doing a discriminate task and you don't need to care about the denominator which is a constant for every element you are going to compare. If you are doing a generative task you need to calculate the exact result of the $P(w|y, \beta)$ and hence to work out the $p(y, \beta)$.

So in your case $$ P(w|y, \beta) = \frac{P(y, w, \beta)}{P(y, \beta)} = P(y|\beta, w) P(w) * \frac{P(\beta)}{P(y, \beta)} \propto P(y|\beta, w) P(w) $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.