2
$\begingroup$

For instance, we observed $n$ tossing of a biased coin with probability of heads being $\theta$. How can I calculate this parameter via maximum likelihood? How can I derive the log-likelihood formula and the correct maximum likelihoo estimate of $\theta$?

$\endgroup$
10
  • 1
    $\begingroup$ Given $\theta$, what is the probability of seeing $x$ out of $n$ heads? So what is the likelihood function for $\theta$ given $x$ out of $n$ heads? How would you find a $\theta$ which maximises this likelihood? $\endgroup$
    – Henry
    May 11 '14 at 17:23
  • $\begingroup$ I have tried to find a solution by myself. But do not know if it correct or not and do not know how tho derive the log-likelihood. $\endgroup$
    – vessilli
    May 11 '14 at 17:23
  • $\begingroup$ Give me moment, I am going to share it here. $\endgroup$
    – vessilli
    May 11 '14 at 17:24
  • 2
    $\begingroup$ You might find this answer to a similar question useful. $\endgroup$
    – QuantIbex
    May 11 '14 at 17:29
  • 1
    $\begingroup$ We know that maximum likelihood is the value of the θ that is most well supported by the data. Let x be the number of heads and n be the number of tails, we get: L(θ,x) = θ^{x} (1-θ)^(n-x) By applying derivative, we get: d/dθ(l(θ,x)) = x/θ - (n-x)/(1-θ), And we get: (1-x/n)θ = (1-θ)x/n ; θ = x/n $\endgroup$
    – vessilli
    May 11 '14 at 17:42
3
$\begingroup$

I've read your comment, where you write $L(\theta;x)=\theta^x(1-\theta)^{n-x}$. This is right for a single toss.

Let's say that head = 1 and tail = 0. If $\mathbf{x}_n$ is a set of $n$ tosses, the number of heads is $\sum_{i=1}^n x_i=n\bar{x}$. The likelihood function is: $$L(\theta;\mathbf{x}_n)\propto \theta^{\sum_ix_i}(1-\theta)^{n-\sum_ix_i}=\theta^{n\bar{x}}(1-\theta)^{n-n\bar{x}}$$ The log-likelihood is: $$\ell(\theta;\mathbf{x}_n)=n\bar{x}\ln\theta+(n-n\bar{x})\ln(1-\theta)$$ and: $$\ell'(\theta;\mathbf{x}_n)=\frac{n\bar{x}}{\theta}-\frac{n-n\bar{x}}{1-\theta}=\frac{n\bar{x}-n\theta}{\theta(1-\theta)}=0\quad\Rightarrow\quad\hat\theta=\bar{x}$$

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.