# Maximum likelihood estimator for a mixture of 2 distributions

Let $$X_1, ..., X_n$$ be iid with one of two PDFs.

If $$\theta = 0$$, then

$$f(x; \theta) = 1, \ 0 < x < 1$$.

if $$\theta = 1$$, then

$$f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$$.

What is the MLE of $$\theta$$?

I can define the likelihood function in terms of indicator functions for $$\theta$$, but then I don't know how to use differentiation to maximize it. Please help.

• Is $\theta$ a parameter? It looks like a latent variable. – Ben Bolker Nov 4 '18 at 1:42
• "Maximize" doesn't automatically imply "differentiate". – Glen_b Nov 4 '18 at 5:33

For all $$0 \leqslant x \leqslant 1$$ the logarithm of the sampling density is:

$$\ln f(x|\theta) = \begin{cases} 0 & & \text{for } \theta=0, \\[6pt] -\ln 2 - \tfrac{1}{2} \ln x & & \text{for } \theta=1. \\[6pt] \end{cases}$$

So you have the log-likelihood function:

$$\ell_\mathbf{x}(\theta) = \begin{cases} 0 & & \text{for } \theta=0, \\[6pt] -n \ln 2 - \tfrac{1}{2} \sum \ln x_i & & \text{for } \theta=1. \\[6pt] \end{cases}$$

This is a binary function (only two possible inputs) and so you maximise it by comparing the outputs for those two inputs. (There is no differentiation involved.) Define $$T(\mathbf{x}) \equiv \tfrac{1}{n} \sum |\ln x_i|$$, which is the average absolute logarithm of the data (this is a sufficient statistic). Maximising this function yields the MLE:

\begin{aligned} \hat{\theta}(\mathbf{x}) &= \mathbb{I} \Big( \ell_\mathbf{x}(1) > \ell_\mathbf{x}(0) \Big) \\[6pt] &= \mathbb{I} \Big( -n \ln 2 - \tfrac{1}{2} \sum \ln x_i > 0 \Big) \\[6pt] &= \mathbb{I} \Big( - \tfrac{1}{2} \sum \ln x_i > n \ln 2 \Big) \\[6pt] &= \mathbb{I} \Big( \tfrac{1}{n} \sum |\ln x_i| > 2 \ln 2 \Big) \\[6pt] &= \mathbb{I} ( T(\mathbf{x}) > 2 \ln 2 ). \\[6pt] \end{aligned}

(Note that the MLE is not uniquely determined for the case where $$T(\mathbf{x}) = 2 \ln 2$$. In this case either parameter value gives the same log-likelihood value. This means that you can use non-strict inequality in the above MLE formula and this still gives a valid MLE.)

• How did you know to maximize $T(x)$?? – MSE Nov 5 '18 at 2:24
• How did you obtain the first line for $\hat{\theta}(x)$? – MSE Nov 5 '18 at 2:25
• @MSE: I have added a new first line to this equation to show you - all I am doing is setting $\hat{\theta} = 1$ if $\ell_\mathbf{x}(1) > \ell_\mathbf{x}(0)$, and setting it to zero otherwise. As to using $T(\mathbf{x})$, that is just a convenient way to express the final result in terms of a simple sufficient statistic. – Ben Nov 5 '18 at 4:20