1
$\begingroup$

I am trying to find the Fisher information of a binomial distribution where $n=2$ and $p=\theta$. I have the log-likelihood function as $$n\ln2 + \sum^{n}_{i=1}x_i\ln \theta + (2n-\sum^{n}_{i=1}x_i)(ln(1-\theta))-\sum^{n}_{i=1}(\ln x_i!+\ln((2-x_i)!))$$I have the second partial derivative of the log-likelihood function as $$\frac{-\sum^{n}_{i=1}x_i}{\theta^2}-\frac{2n-\sum^{n}_{i=1}x_i}{(1-\theta)^2}$$

When I go to take the negative expectation of this, I get $$\frac{n}{\theta}+\frac{2n-n\theta}{(1-\theta)^2}$$ which simplifies to $\frac{n}{\theta(1-\theta)^2}$. I know that the Fisher information is supposed to be $\frac{n}{\theta(1-\theta)}$, but I cannot figure out what I'm doing wrong that gets rid of the square. I think it has to do with the $2n$ somewhere along the line, but I cannot figure out where. Does my log-likelihood function look right? What about the negative expectation?

$\endgroup$
4
  • 1
    $\begingroup$ 1. You can drop both of the terms that don't involve $\theta$ since they will disappear when you take the first derivative anyway. 2. Why does your third term in the first equation have $2n$? If $n=2$, then that should just be $2$. $\endgroup$
    – jbowman
    Commented Mar 20 at 3:57
  • $\begingroup$ Related: stats.stackexchange.com/a/602524/20519 $\endgroup$
    – Zhanxiong
    Commented Mar 20 at 4:20
  • $\begingroup$ Cross-posted at math.stackexchange.com/q/4884056/321264. $\endgroup$ Commented Mar 20 at 10:53
  • 1
    $\begingroup$ It's not clear from your question (or your notation) what the exact setup is. Do you have a single observation $X \sim \mathrm{Bin}(n,\theta)$ where $n=1$ or a random sample $X_1,\ldots,X_n \sim \mathrm{Bin}(2, \theta)$? $\endgroup$ Commented Mar 20 at 20:08

1 Answer 1

0
$\begingroup$

So for a binomial distribution with $n$ trials $x_1,...,x_n$ and probability $\theta$ we have the likelihood

$$L(X|\theta)=\binom{n}{\sum x_i}\theta^{\sum x_i}(1-\theta)^{(n-\sum x_i)}$$

log-likelihood:

$$l=\log\binom{n}{\sum x_i}+\sum x_i\log\theta+(n-\sum x_i)\log(1-\theta)$$

First derivative:

$$\frac{\partial}{\partial\theta}l=\frac{\sum x_i}{\theta}-\frac{n-\sum x_i}{1-\theta}$$

Second derivative:

$$\frac{\partial^2}{\partial\theta^2}l=-\frac{\sum x_i}{\theta^2}-\frac{n-\sum x_i}{(1-\theta)^2}=\frac{-(1-\theta)^2\sum x_i-\theta^2(n-\sum x_i)}{\theta^2(1-\theta)^2}=\frac{-\sum x_i+2\theta\sum x_i-\theta^2\sum x_i-\theta^2n+\theta^2\sum x_i}{\theta^2(1-\theta)^2}=\frac{-(1-2\theta)\sum x_i-\theta^2n}{\theta^2(1-\theta)^2}$$

Information:

$$I(\theta)=-E\left[ \frac{\partial^2}{\partial\theta^2}l \right]=-E\left[\frac{-(1-2\theta)\sum x_i-\theta^2n}{\theta^2(1-\theta)^2}\right]=\frac{(1-2\theta)E\left[\sum x_i\right]+\theta^2n}{\theta^2(1-\theta)^2}=\frac{(1-2\theta)\theta n+\theta^2n}{\theta^2(1-\theta)^2}=\frac{\theta n-2\theta^2n+\theta^2n}{\theta^2(1-\theta)^2}=\frac{\theta n(1-\theta)}{\theta^2(1-\theta)^2}=\frac{n}{\theta(1-\theta)}.\blacksquare$$

$\endgroup$
4
  • $\begingroup$ The likelihood in the question has terms of the form $x_i!$ and $(2-x_i)!$, which seems to suggest that the sample is from a $\mathrm{Bin}(2,\theta)$, not a $\mathrm{Bernoulli}(\theta)$ (as in your answer), but it's not entirely clear. $\endgroup$ Commented Mar 20 at 20:11
  • $\begingroup$ As I understand the question, these terms come from the binomial coefficient and correspond to the assumption $n=2$ $\endgroup$
    – Spätzle
    Commented Mar 21 at 5:31
  • $\begingroup$ Then what does $n$ represent in the upper limit of the sum? $\endgroup$ Commented Mar 21 at 7:33
  • $\begingroup$ You sum over the different $x_i$ values. It seems as if the OP has some serious notation issues there (indexing using $x$ for some reason), fixed it now. Eagerly expecting their response regarding you setup comment, which might be crucial here. It is indeed very unclear. $\endgroup$
    – Spätzle
    Commented Mar 21 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.