Mean field variational inference

In Chris Bishop PRML book p.465 equation 10.6, the derivation doesn't explain why exactly the term $\int q_j ln(q_j) dz_j$ was generated, is not that term supposed to be multiplied by constant, did the derivation supposes that all the values of $q_i \forall i \neq j$ are constants, if that is the case then why the equation lists the product $\prod_{i \neq j} q_i dz_i$ as not a constant and lists it as a function.

• You'll need to provide a lot more information for this to be answerable. What book by Bishop? Eg, can you paste in the context for this? Etc. Dec 19, 2014 at 19:37
• @gung Thanks a lot, by PRML I was referring to his book (Pattern recognition and machine learning) [research.microsoft.com/en-us/um/people/cmbishop/PRML/] Dec 19, 2014 at 19:56

The equation lists $\prod_{i\not=j} q_i dz_i$ as not a constant because it's a multiplicative factor to $q_i$ and is important from optimization perspective. Further down a formula (10.9) for $q_i$ update is derived and it heavily depends on these $\{q_j\}_{j\not=i}$.
From the other hand, $\int q_i \sum_{i\not=j} \ln q_j$ is left aside as $const$ because it's independent of concrete $q_i$ (as long as it's a probability distribution).
• Thanks a lot, a final question: why is $\int q_i \sum_{i\not=j} \ln q_j$ a constant even though it contains $q_i$ Dec 19, 2014 at 20:01
• @Magellanea because of independence assumption. $q_j$ is independent of $q_i$ (for $i \not= j$), thus sum can be moved out of integral and, since $q_i$ is valid probability density function, it integrates to one. Dec 19, 2014 at 20:02
• so to recap, away from $j$ that we optimize for all other $i \neq j$ are taken as constant vals, and their use in the product $\prod_{i \neq j}$ is for using in formula (10.9) Dec 19, 2014 at 20:08
• Formula (10.9) is derived from (10.6) by minimizing it. You can mark the product as a constant, but you can't throw it away because the function inside of integral depends on it. On the contrary, sum can be thrown away because it's independent of $q_j$ Dec 19, 2014 at 20:19