Variational Inference of Univariate Gaussian mixtures

Question

I am reading this paper. In the paper, they use an example of mixture of unit-variance univariate Gaussians with the following parameterization:

\begin{align} \mu_k & \sim \mathcal{N}(0, \sigma^2) \\ c_i & \sim categorical(\frac{1}{K}) \\ x_i~\vert ~c_i, \mathbf{\mu} & \sim \mathcal{N}(\mu_{c_i}, 1) \end{align}

Then, mean field variational family is introduced for latent variables $\mu$ and $c$ in the following form:

\begin{align} q(\mathbf{\mu}, \mathbf{c}) &= \prod_{k} q(\mu_k; m_k, s_k^2) \prod_{i}q(c_i; \phi_i) \end{align}

My question is that why we need to introduce variational distribution for $\mu$'s? In the probabilistic model, we have already assumed that they come from a normal distribution with 0 mean and $\sigma^2$ variance, which is a hyperparameter.

Without that, we can still derive the optimal solutions for $\phi$'s first, and then $\mu$'s by taking the partial derivatives from ELBO, right? I am a bit confused.

Answer 1

So I emailed Prof. David Blei and he responded me as follows:

the prior and posterior of mu are different. the model's distribution is the prior. the variational distribution seeks to approximate its posterior.