3
$\begingroup$

Imagine that we have mixture of two normal distributions with mixture parameter $\theta$:

$$p(y_i|\theta) = \theta\phi(y_i;\mu_1, \sigma_1^2) + (1 - \theta)\phi(y_i; \mu_2, \sigma_2^2)$$

Assume that the ONLY unknown parameter is $\theta$. I want to get the MLE estimate of $\theta$. Here is what I do:

$$L(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) = \prod_{i = 1}^{n} p(y_i|\theta) = \prod_{i = 1}^{n} \Big(\theta\phi(y_i;\mu_1, \sigma_1^2) + (1 - \theta)\phi(y_i; \mu_2, \sigma_2^2)\Big) \Rightarrow$$ $$L(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) \propto \prod_{i = 1}^{n}\Big[ \theta \Big(\phi_{i1} - \phi_{i2}\Big)\Big] = \theta^n \prod_{i = 1}^{n}\Big(\phi_{i1} - \phi_{i2}\Big) = \theta^nG(Y, param)$$

where $\phi_{i1}$ refers to the first normal density with $y_i$ plugged in and $\phi_{i2}$ refers to the second normal density. Note that there is no $\theta$ involved in G(Y, param).

Now, in order to get the MLE, I take log and then derivative:

$$l(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) = log(L(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2)) \propto n log(\theta) + log(G) \Rightarrow $$

$$l'(\theta | data, \mu_1, \sigma_1^2, \mu_2, \sigma_2^2) = \frac{n}{\theta}$$

If my calculations above is right, then it's a very weird derivative and I don't know how to get the MLE estimate.

$\endgroup$
5
$\begingroup$

The problem is in the factorisation after "$\Rightarrow$" since

$$\theta \phi_{i1} +(1-\theta)\phi_{i2} = \theta(\phi_{i1}-\phi_{i2})+\phi_{i2}.$$

Then, the term $\phi_{i2}$ cannot be eliminated using an argument of proportionality and it has to be considered in the product. This produces a different likelihood and the corresponding estimator can also be found using the log-likelihood.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.