Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

It is well-known that maximum likelihood estimate for a mixture model, with the mixture distributions known, and the estimation is done for the mixture coefficients is consistent (I think) -- the ML objective has a single likelihood, so expectation-maximization will be consistent.

What if the mixture distributions themselves are estimated from data, and their estimators are consistent? Will that yield a consistent estimator for estimating the mixture components as well, when using the estimated mixture distributions, or not?

(To put it more mathematically:

Say we have a discrete random variable $X$ accepting values between $1$ and $n$.

We also have $m$ fixed distributions $p_i(X = x)$ for $i = 1, ..., m$.

We want to estimate $\theta$ for the model $$p(X = x) = \sum_i \theta_i p_i(X = x).$$

The maximum likelihood estimate is going to be consistent.

Now what if we didn't have $p_i$ but instead $\hat{p_i}$ (estimators of $p_i$) such that $\hat{p_i}(X = x) \rightarrow p_i(X = x)$ as we have more data?

I am guessing the ML estimator for $\theta$ while using $\hat{p_i}$ will still be consistent, is that correct? If so, what is roughly the way to show it?


share|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.