Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

It is well-known that maximum likelihood estimate for a mixture model, with the mixture distributions known, and the estimation is done for the mixture coefficients is consistent (I think) -- the ML objective has a single likelihood, so expectation-maximization will be consistent.

What if the mixture distributions themselves are estimated from data, and their estimators are consistent? Will that yield a consistent estimator for estimating the mixture components as well, when using the estimated mixture distributions, or not?

(To put it more mathematically:

Say we have a discrete random variable $X$ accepting values between $1$ and $n$.

We also have $m$ fixed distributions $p_i(X = x)$ for $i = 1, ..., m$.

We want to estimate $\theta$ for the model $$p(X = x) = \sum_i \theta_i p_i(X = x).$$

The maximum likelihood estimate is going to be consistent.

Now what if we didn't have $p_i$ but instead $\hat{p_i}$ (estimators of $p_i$) such that $\hat{p_i}(X = x) \rightarrow p_i(X = x)$ as we have more data?

I am guessing the ML estimator for $\theta$ while using $\hat{p_i}$ will still be consistent, is that correct? If so, what is roughly the way to show it?

)

share|improve this question
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.