For instance, we observed $n$ tossing of a biased coin with probability of heads being $\theta$. How can I calculate this parameter via maximum likelihood? How can I derive the log-likelihood formula and the correct maximum likelihoo estimate of $\theta$?
10
-
1$\begingroup$ Given $\theta$, what is the probability of seeing $x$ out of $n$ heads? So what is the likelihood function for $\theta$ given $x$ out of $n$ heads? How would you find a $\theta$ which maximises this likelihood? $\endgroup$– HenryMay 11, 2014 at 17:23
-
$\begingroup$ I have tried to find a solution by myself. But do not know if it correct or not and do not know how tho derive the log-likelihood. $\endgroup$– vessilliMay 11, 2014 at 17:23
-
$\begingroup$ Give me moment, I am going to share it here. $\endgroup$– vessilliMay 11, 2014 at 17:24
-
2$\begingroup$ You might find this answer to a similar question useful. $\endgroup$– QuantIbexMay 11, 2014 at 17:29
-
1$\begingroup$ We know that maximum likelihood is the value of the θ that is most well supported by the data. Let x be the number of heads and n be the number of tails, we get: L(θ,x) = θ^{x} (1-θ)^(n-x) By applying derivative, we get: d/dθ(l(θ,x)) = x/θ - (n-x)/(1-θ), And we get: (1-x/n)θ = (1-θ)x/n ; θ = x/n $\endgroup$– vessilliMay 11, 2014 at 17:42
|
Show 5 more comments
1 Answer
$\begingroup$
$\endgroup$
0
I've read your comment, where you write $L(\theta;x)=\theta^x(1-\theta)^{n-x}$. This is right for a single toss.
Let's say that head = 1 and tail = 0. If $\mathbf{x}_n$ is a set of $n$ tosses, the number of heads is $\sum_{i=1}^n x_i=n\bar{x}$. The likelihood function is: $$L(\theta;\mathbf{x}_n)\propto \theta^{\sum_ix_i}(1-\theta)^{n-\sum_ix_i}=\theta^{n\bar{x}}(1-\theta)^{n-n\bar{x}}$$ The log-likelihood is: $$\ell(\theta;\mathbf{x}_n)=n\bar{x}\ln\theta+(n-n\bar{x})\ln(1-\theta)$$ and: $$\ell'(\theta;\mathbf{x}_n)=\frac{n\bar{x}}{\theta}-\frac{n-n\bar{x}}{1-\theta}=\frac{n\bar{x}-n\theta}{\theta(1-\theta)}=0\quad\Rightarrow\quad\hat\theta=\bar{x}$$
