I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models.
For example, say I have a data set on which I run a Dirichlet Process Mixture Model (using Gibbs/MCMC) and the result is a posterior distribution over number of clusters, and parameter estimates for each cluster. Say the mode of the posterior is $k=10$ clusters. Ideally, in practice, this would be my best guess at the number of clusters.
Now suppose I instead fit a finite mixture model. Let's say I fit $k$ Gaussian Mixture Models (using EM) by looping from $k = 1...n$ (where $n$ is the number of data points). Obviously this is time intensive and inefficient but the point is that I want to find the best $k$. My results may slightly change per run, but imagine I do this many times and find some optimal $k$, on average.
Two questions:
Is the $k$ that I should find using both approaches going to be similar? Intuitively it should be since I'm finding some number of Gaussian clusters using each method that fits the data, but one uses Gibbs/MCMC and the other EM. It's not obvious to me that they should be the same since I'm not sure they are "maximizing" the same density per Gibbs/EM iteration.
Are the parameters of the resulting $k$ Gaussians in each model going to be roughly equivalent? Again, intuitively each model should find the location of the $k$ cluster centers and their spread.
Is there any reason (mathematical explanation preferred) why these two methods should give very different results?