Variational Inference - deriving coordinate update equations for mixture model

Question

I'm currently going through this paper by Blei et. al. that describes the setup and derivation of the coordinate ascent equations for a Gaussian mixture model with K components. I am having some trouble with the coordinate ascent update for the mixture assignment probabilities.

Keeping in line with the paper's notation, we have that

$\mu \sim \text{N}(0, \sigma^2)$

$c \sim \text{Multinomial}(1, [\frac{1}{K}, \ldots \frac{1}{K}]) $

$x|\mu, c \sim \text{N}(c^T\mu, 1)$

The paper posits a multinomial as the variational distribution of the mixture assignments $c_i$. The optimal update for the mixture assignments is then given by

$q^{*}(c_i) \propto exp(\mathbb{E}_{-j}[log\, p(\mathbf{x}, \mathbf{c}, \mu)])$

$\propto exp(log \, p(c_i) + \mathbb{E}[log \, p(x_i|c_i, \mu)])$

$\propto exp(\mathbb{E}[log\, p(x_i | c_i, \mu)]) $

Where the expectation is taken with respect to $q(\mu)$. The inner term is equal to

$\mathbb{E}[log\, p(x_i | c_i, \mu)] = \sum_{k=1}^{k} c_{ik}\mathbb{E}[log\, p(x_i|\mu_k)]$

$\propto \sum_{k} c_{ik} \mathbb{E}[-\frac{(x_i-\mu_{k})^2}{2\sigma^2}]$

$\propto \sum_{k} c_{ik}(\mathbb{E}[\mu_k]x_i - \mathbb{E}[\mu_k^2]/2)$

From this, the authors conclude that the update for the multinomial variational parameter is

$P_q(c_{ik} = 1) = \psi_{ik} \propto exp(\mathbb{E}[\mu_k]x_i - \mathbb{E}[\mu_k^2]/2)$

I am struggling with the last step of the derivation. The second to last step had an extra $c_{ik}$ term that seemed to just disappear, and I can't find out how they were able to cancel it out.

Answer 1

$c_i$ is an "indicator vector". This means that $c_{ik}$ is 1 only when $x_i$ belongs to cluster $k$, and 0 otherwise. Thus when we are considering $\psi_{ik}$, we can directly evaluate the expression $$\exp\left(\mathbb{E}[\mu_k]x_i - \frac{\mathbb{E}[\mu_k^2]}2\right)$$ for that particular $k$.