3
$\begingroup$

In Chris Bishop PRML book p.465 equation 10.6, the derivation doesn't explain why exactly the term $\int q_j ln(q_j) dz_j $ was generated, is not that term supposed to be multiplied by constant, did the derivation supposes that all the values of $q_i \forall i \neq j$ are constants, if that is the case then why the equation lists the product $\prod_{i \neq j} q_i dz_i$ as not a constant and lists it as a function.

$\endgroup$
4
$\begingroup$

The equation lists $\prod_{i\not=j} q_i dz_i$ as not a constant because it's a multiplicative factor to $q_i$ and is important from optimization perspective. Further down a formula (10.9) for $q_i$ update is derived and it heavily depends on these $\{q_j\}_{j\not=i}$.

From the other hand, $\int q_i \sum_{i\not=j} \ln q_j$ is left aside as $const$ because it's independent of concrete $q_i$ (as long as it's a probability distribution).

$\endgroup$
  • $\begingroup$ Thanks a lot, a final question: why is $\int q_i \sum_{i\not=j} \ln q_j$ a constant even though it contains $q_i$ $\endgroup$ – Magellanea Dec 19 '14 at 20:01
  • $\begingroup$ @Magellanea because of independence assumption. $q_j$ is independent of $q_i$ (for $i \not= j$), thus sum can be moved out of integral and, since $q_i$ is valid probability density function, it integrates to one. $\endgroup$ – Artem Sobolev Dec 19 '14 at 20:02
  • $\begingroup$ so to recap, away from $j$ that we optimize for all other $i \neq j$ are taken as constant vals, and their use in the product $\prod_{i \neq j}$ is for using in formula (10.9) $\endgroup$ – Magellanea Dec 19 '14 at 20:08
  • $\begingroup$ Formula (10.9) is derived from (10.6) by minimizing it. You can mark the product as a constant, but you can't throw it away because the function inside of integral depends on it. On the contrary, sum can be thrown away because it's independent of $q_j$ $\endgroup$ – Artem Sobolev Dec 19 '14 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.