My question:

I usually don't have much trouble with deriving the MLE from the log-likelihood, but I am a little stumped on an example I found in "In All Likelihood" [1] pg.75.

Could someone show the steps from the log-likelihood to the MLE?

What the book says: Discrete data are usually presented in grouped form. For example, suppose $x_1, ..., x_N$ are iid sample from binomial($n,\theta$) with $n$ known. We first summarize the data where

$$ \begin{array}{c|c c c c} k& 0 & 1 & \dots &n \\ \hline n_k& n_0&n_1 &\dots &n_n\\ \end{array} $$

where $n_k$ is the number of $x_i$'s equal to $k$, so $\sum_k n_k = N$. We can now think of the data ($n_0,\ldots,n_n$) as having a multinomial distribution with probabilities ($p_0,\dots,p_n$) given by the binomial probabilities

$$ p_k = {n\choose k} \theta^k(1-\theta)^{n-k} $$

The log-likelihood is given by

$$ \log L(\theta)= \sum_{k=0}^n n_k\log p_k. $$

We can show that the MLE is $\theta$ is

$$ \hat{\theta} = \frac{\sum_k kn_k}{Nn} $$

with standard error

$$ \operatorname{se}(\hat{\theta}) = \sqrt{\frac{\hat{\theta}(1-\hat{\theta})}{Nn}} $$


[1] Y. Pawitan, (2001), 'In All Likelihood: Statistical Modelling and Inference Using Likelihood', Oxford University Press.

  • 1
    Who are you quoting at the start (the part in grey at the top of your post using blockquote markup)? Or was that intended an a form of highlighting? How does the part under that grey part relate to your question? It sounds like you're trying to answer a question... What are you seeking in relation to that? – Glen_b Mar 4 '17 at 4:53
  • 2
    Please. Don't write \text{log }. Write \log. With \log You automatically get proper spacing in expressions like $a\log b$ and $a\log(b)$ (coded as a\log b and a\log(b)). You don't need to add spacing manually. – Michael Hardy Mar 4 '17 at 5:22
  • Do you mean All of Statistics?! – Xi'an Mar 4 '17 at 10:27
  • @Glen_b The quote was my question. The rest of the question is a direct quote from the book. I tried to quote the part from the book, but it messed up my latex table formatting. So in order to separate my question from the book I re-purposed the quote tool to be my question while the direct quote from the book was unquoted. – Alex Mar 5 '17 at 4:00
  • 1
    If you expect people to read what you marked using quote markup to be your question and what is unmarked to be a quote you should explain what you're doing. By doing it backwards it's confusing without an explanation in the question. I have made some changes to so indicate (as well as added the reference). You can edit further if you wish but please make sure some form of explanation of your unusual formatting remains. – Glen_b Mar 5 '17 at 6:14
up vote 1 down vote accepted

I think I just figured this out.

$$ \log L(\theta) = \sum_{k=0}^n n_k\log p_k \\ = \sum_{k=0}^n n_k(k\log(\theta) + (n-k)\log(1-\theta)) \\ \frac{\partial}{\partial\theta} = \frac{\sum_{k=0}^n n_kk}{\theta} - \frac{\sum_{k=0}^n n_k(n-k)}{1-\theta} \\ 0 = \frac{\sum_{k=0}^n n_k k}{\theta} - \frac{\sum_{k=0}^n n_k(n-k)}{1-\theta} \\ \frac{\sum_{k=0}^n n_k(n-k)}{1-\theta} = \frac{\sum_{k=0}^n n_k k}{\theta} \\ \frac{\sum_{k=0}^n n_k(n-k)}{\sum_{k=0}^n n_k k} = \frac{1-\theta}{\theta} \\ \frac{\sum_{k=0}^n n_kn}{\sum_{k=0}^n n_k k} - 1 = \frac{1}{\theta}-1 \\ \frac{\sum_{k=0}^n n_kn}{\sum_{k=0}^n n_k k} = \frac{1}{\theta} \\ \theta = \frac{\sum_{k=0}^n n_k k}{\sum_{k=0}^n n_kn} \\ \theta = \frac{\sum_{k=0}^n n_k k}{Nn} $$

  • Please look at my comment under the question. I edited your answer accordingly. – Michael Hardy Mar 4 '17 at 5:23

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.