My question:

I usually don't have much trouble with deriving the MLE from the log-likelihood, but I am a little stumped on an example I found in "In All Likelihood" [1] pg.75.

Could someone show the steps from the log-likelihood to the MLE?

What the book says: Discrete data are usually presented in grouped form. For example, suppose $x_1, ..., x_N$ are iid sample from binomial($n,\theta$) with $n$ known. We first summarize the data where

$$ \begin{array}{c|c c c c} k& 0 & 1 & \dots &n \\ \hline n_k& n_0&n_1 &\dots &n_n\\ \end{array} $$

where $n_k$ is the number of $x_i$'s equal to $k$, so $\sum_k n_k = N$. We can now think of the data ($n_0,\ldots,n_n$) as having a multinomial distribution with probabilities ($p_0,\dots,p_n$) given by the binomial probabilities

$$ p_k = {n\choose k} \theta^k(1-\theta)^{n-k} $$

The log-likelihood is given by

$$ \log L(\theta)= \sum_{k=0}^n n_k\log p_k. $$

We can show that the MLE is $\theta$ is

$$ \hat{\theta} = \frac{\sum_k kn_k}{Nn} $$

with standard error

$$ \operatorname{se}(\hat{\theta}) = \sqrt{\frac{\hat{\theta}(1-\hat{\theta})}{Nn}} $$

[1] Y. Pawitan, (2001), 'In All Likelihood: Statistical Modelling and Inference Using Likelihood', Oxford University Press.

  • 1
    $\begingroup$ Who are you quoting at the start (the part in grey at the top of your post using blockquote markup)? Or was that intended an a form of highlighting? How does the part under that grey part relate to your question? It sounds like you're trying to answer a question... What are you seeking in relation to that? $\endgroup$
    – Glen_b
    Commented Mar 4, 2017 at 4:53
  • 3
    $\begingroup$ Please. Don't write \text{log }. Write \log. With \log You automatically get proper spacing in expressions like $a\log b$ and $a\log(b)$ (coded as a\log b and a\log(b)). You don't need to add spacing manually. $\endgroup$ Commented Mar 4, 2017 at 5:22
  • $\begingroup$ Do you mean All of Statistics?! $\endgroup$
    – Xi'an
    Commented Mar 4, 2017 at 10:27
  • $\begingroup$ @Glen_b The quote was my question. The rest of the question is a direct quote from the book. I tried to quote the part from the book, but it messed up my latex table formatting. So in order to separate my question from the book I re-purposed the quote tool to be my question while the direct quote from the book was unquoted. $\endgroup$
    – Alex
    Commented Mar 5, 2017 at 4:00
  • 1
    $\begingroup$ If you expect people to read what you marked using quote markup to be your question and what is unmarked to be a quote you should explain what you're doing. By doing it backwards it's confusing without an explanation in the question. I have made some changes to so indicate (as well as added the reference). You can edit further if you wish but please make sure some form of explanation of your unusual formatting remains. $\endgroup$
    – Glen_b
    Commented Mar 5, 2017 at 6:14

1 Answer 1


I think I just figured this out.

$$ \log L(\theta) = \sum_{k=0}^n n_k\log p_k \\ = \sum_{k=0}^n n_k(k\log(\theta) + (n-k)\log(1-\theta)) \\ \frac{\partial}{\partial\theta} = \frac{\sum_{k=0}^n n_kk}{\theta} - \frac{\sum_{k=0}^n n_k(n-k)}{1-\theta} \\ 0 = \frac{\sum_{k=0}^n n_k k}{\theta} - \frac{\sum_{k=0}^n n_k(n-k)}{1-\theta} \\ \frac{\sum_{k=0}^n n_k(n-k)}{1-\theta} = \frac{\sum_{k=0}^n n_k k}{\theta} \\ \frac{\sum_{k=0}^n n_k(n-k)}{\sum_{k=0}^n n_k k} = \frac{1-\theta}{\theta} \\ \frac{\sum_{k=0}^n n_kn}{\sum_{k=0}^n n_k k} - 1 = \frac{1}{\theta}-1 \\ \frac{\sum_{k=0}^n n_kn}{\sum_{k=0}^n n_k k} = \frac{1}{\theta} \\ \theta = \frac{\sum_{k=0}^n n_k k}{\sum_{k=0}^n n_kn} \\ \theta = \frac{\sum_{k=0}^n n_k k}{Nn} $$

  • 1
    $\begingroup$ Please look at my comment under the question. I edited your answer accordingly. $\endgroup$ Commented Mar 4, 2017 at 5:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.