# Mean field variational inference

In Chris Bishop PRML book p.465 equation 10.6, the derivation doesn't explain why exactly the term $\int q_j ln(q_j) dz_j$ was generated, is not that term supposed to be multiplied by constant, did the derivation supposes that all the values of $q_i \forall i \neq j$ are constants, if that is the case then why the equation lists the product $\prod_{i \neq j} q_i dz_i$ as not a constant and lists it as a function.

The equation lists $\prod_{i\not=j} q_i dz_i$ as not a constant because it's a multiplicative factor to $q_i$ and is important from optimization perspective. Further down a formula (10.9) for $q_i$ update is derived and it heavily depends on these $\{q_j\}_{j\not=i}$.
From the other hand, $\int q_i \sum_{i\not=j} \ln q_j$ is left aside as $const$ because it's independent of concrete $q_i$ (as long as it's a probability distribution).
• Thanks a lot, a final question: why is $\int q_i \sum_{i\not=j} \ln q_j$ a constant even though it contains $q_i$ – Magellanea Dec 19 '14 at 20:01
• @Magellanea because of independence assumption. $q_j$ is independent of $q_i$ (for $i \not= j$), thus sum can be moved out of integral and, since $q_i$ is valid probability density function, it integrates to one. – Artem Sobolev Dec 19 '14 at 20:02
• so to recap, away from $j$ that we optimize for all other $i \neq j$ are taken as constant vals, and their use in the product $\prod_{i \neq j}$ is for using in formula (10.9) – Magellanea Dec 19 '14 at 20:08
• Formula (10.9) is derived from (10.6) by minimizing it. You can mark the product as a constant, but you can't throw it away because the function inside of integral depends on it. On the contrary, sum can be thrown away because it's independent of $q_j$ – Artem Sobolev Dec 19 '14 at 20:19