Gaussian Naive Bayes really equivalent to GMM with diagonal covariance matrices? Murphy writes that a multivariate Gaussian used in a generative classifier ("Gaussian discriminant analysis"), i.e.,
$p(\mathbf x|y=c,\mathbf\theta) = \mathcal{N}(\mathbf x|\mathbf y_c,\mathbf \Sigma_c)$, is equivalent to Gaussian Naive Bayes if $\mathbf \Sigma_c$ is diagonal. This makes sense to me since GNB assumes independent features, each following a normal distribution.
However, I've also read in multiple papers, for example this one, about an alleged equivalence of Naive Bayes and a mixture of Gaussians with diagonal covariance matrices. How can this be the case when a mixture can approximate any distribution given enough Gaussian base components? It seems to me that the GMM is more powerful than both Naive Bayes and Gaussian discriminant analysis in that it can provide better likelihoods.
 A: In the second paper you linked to:

Naive Bayes represents a distribution as a mixture of components, where within each component all variables are assumed independent of each other. Given enough components, it can approximate an arbitrary distribution arbitrarily closely. ... When learned from data, naive Bayes models never contain more components than there are examples in the data ...

and then later, the quote you mentioned,

Naive Bayes models can be viewed as Bayesian networks in which each $X_i$ has $C$ as the sole parent and $C$ has no parents. A naive Bayes model with Gaussian $P(X_i |C )$ is equivalent to a mixture of Gaussians with diagonal covariance matrices (Dempster et al., 1977).

Then they go on to show that, in the discrete case, $P(X=x)=\sum_{c=1}^k\left(P(c)\prod_{i=1}^{|X|}P(x_i|c)\right)$.
So consider $X\sim N(\mu,\Sigma^\text{diag})$. It's easy to see that:
$$
P(X=x)=\sum_{c=1}^kP(c)\prod_{i=1}^{|X|}P(x_i|c)=\sum_{c=1}^kP(c)\cdot P(x_1,\dots,x_{|X|}|c)
$$
since the joint distribution of $X$ is just the product of the distributions of its independent components. This, of course, is a mixture of $k$ Gaussians, each with weight $P(c)$. So unless they're talking about something else, I think I was right in the comments. What makes me less than 100% confident is that a) they talk about "enough components" which makes me think it should be more than $k$, and b) I have no idea where in Dempster (1977) this is or why they cite that paper of all things. I tried sifting through Dempster but it's on a completely different subject (E-M algorithm for ML with missing data) and I don't think they even had the term "Naive Bayes" back then. APA really needs to start requiring page numbers somehow.
I think the reason they point this out is that they develop Naive Bayes from a graphical / network perspective, so this provides a different intuition.
A: (Related): Quoting Greg Heath from a Google Groups discussion,

In general, the two concepts (Gaussian Mixture Model and Naive Bayes Network) are not related.
The Gaussian Mixture Model approximates the probability density with a
  sum of Gaussians. Each gaussian is, in general, characterized by a
  full covariance matrix. The variables are independent if, and only if,
  the matrix is diagonal.
The Naive Bayes Assumption assumes that the variables are independent.
  Therefore, the probability density can be characterized by a product
  of univariate densities. If the univariate densities are Gaussian, the
  probability is characterized by a Gaussian Mixture Model with a
  diagonal covariance matrix.
Hope this helps.

