Let $X_1, ..., X_n$ be iid with one of two PDFs.

If $\theta = 0$, then

$f(x; \theta) = 1, \ 0 < x < 1$.

if $\theta = 1$, then

$f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$.

What is the MLE of $\theta$?

I can define the likelihood function in terms of indicator functions for $\theta$, but then I don't know how to use differentiation to maximize it. Please help.

  • $\begingroup$ Is $\theta$ a parameter? It looks like a latent variable. $\endgroup$
    – Ben Bolker
    Nov 4 '18 at 1:42
  • 2
    $\begingroup$ "Maximize" doesn't automatically imply "differentiate". $\endgroup$
    – Glen_b
    Nov 4 '18 at 5:33

For all $0 \leqslant x \leqslant 1$ the logarithm of the sampling density is:

$$\ln f(x|\theta) = \begin{cases} 0 & & \text{for } \theta=0, \\[6pt] -\ln 2 - \tfrac{1}{2} \ln x & & \text{for } \theta=1. \\[6pt] \end{cases}$$

So you have the log-likelihood function:

$$\ell_\mathbf{x}(\theta) = \begin{cases} 0 & & \text{for } \theta=0, \\[6pt] -n \ln 2 - \tfrac{1}{2} \sum \ln x_i & & \text{for } \theta=1. \\[6pt] \end{cases}$$

This is a binary function (only two possible inputs) and so you maximise it by comparing the outputs for those two inputs. (There is no differentiation involved.) Define $T(\mathbf{x}) \equiv \tfrac{1}{n} \sum |\ln x_i|$, which is the average absolute logarithm of the data (this is a sufficient statistic). Maximising this function yields the MLE:

$$\begin{equation} \begin{aligned} \hat{\theta}(\mathbf{x}) &= \mathbb{I} \Big( \ell_\mathbf{x}(1) > \ell_\mathbf{x}(0) \Big) \\[6pt] &= \mathbb{I} \Big( -n \ln 2 - \tfrac{1}{2} \sum \ln x_i > 0 \Big) \\[6pt] &= \mathbb{I} \Big( - \tfrac{1}{2} \sum \ln x_i > n \ln 2 \Big) \\[6pt] &= \mathbb{I} \Big( \tfrac{1}{n} \sum |\ln x_i| > 2 \ln 2 \Big) \\[6pt] &= \mathbb{I} ( T(\mathbf{x}) > 2 \ln 2 ). \\[6pt] \end{aligned} \end{equation}$$

(Note that the MLE is not uniquely determined for the case where $T(\mathbf{x}) = 2 \ln 2$. In this case either parameter value gives the same log-likelihood value. This means that you can use non-strict inequality in the above MLE formula and this still gives a valid MLE.)

  • $\begingroup$ How did you know to maximize $T(x)$?? $\endgroup$
    – MSE
    Nov 5 '18 at 2:24
  • $\begingroup$ How did you obtain the first line for $\hat{\theta}(x)$? $\endgroup$
    – MSE
    Nov 5 '18 at 2:25
  • $\begingroup$ @MSE: I have added a new first line to this equation to show you - all I am doing is setting $\hat{\theta} = 1$ if $\ell_\mathbf{x}(1) > \ell_\mathbf{x}(0)$, and setting it to zero otherwise. As to using $T(\mathbf{x})$, that is just a convenient way to express the final result in terms of a simple sufficient statistic. $\endgroup$
    – Ben
    Nov 5 '18 at 4:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.