1
$\begingroup$

I have the log-likelihood function:

$$l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$

And I need to calculate the maximum likelihood estimator of $p_i$. When I do this, for some reason when differentiating, the summation sign vanishes. Why is this?

EDIT: To calculate the maximum likelihood estimators, I would differentiate my log likelihood, equal that to 0 and solve:

$$ \frac{\partial{l}}{\partial{p_i}} = \sum_{i = 1}^n \left( \frac{1}{p_i} - \frac{y_i}{1 - p_i} \right) = 0 $$

Rearranging gives me

$$ \sum \frac{1}{p_i} = \sum \left( \frac{y_i}{1 - p_i} \right) $$

Oh, does this now become

$$\frac{n}{p_i} = \frac{n y_i}{ 1 - pi} $$

And then I can divide through by n and rearrange to get my value for $\hat{p_i}$?

$\endgroup$
6
  • $\begingroup$ It would help if you showed the steps you are taking in calculating this. $\endgroup$
    – Peter Flom
    Commented Dec 25, 2012 at 17:57
  • 2
    $\begingroup$ Say that you take the derivative with respect to $p_1$ The derivative "can enter the summation", by linearity. Now, $p_1$ appears in the first term ($i = 1$) only. The derivative of all other terms ($i \neq 1$) are thus $0$. This is why the summation sign disappears. $\endgroup$
    – ocram
    Commented Dec 25, 2012 at 18:01
  • $\begingroup$ But in some of my other questions, I've had to leave the summation sign in to solve for my estimator. And would that not only give me the estimator for $p_1$ and not $p_i$? $\endgroup$
    – Kaish
    Commented Dec 25, 2012 at 18:05
  • 2
    $\begingroup$ What I said is still valid for any $i$ between $1$ and $n$. $p_i$ appears in the $i$th term only; all other terms do not contribute to the derivative. Your first equation in your edit is thus wrong. $\endgroup$
    – ocram
    Commented Dec 25, 2012 at 18:08
  • $\begingroup$ So if I'm doing it for $i = i$, my first equation should read the same, just without the summation sign? $\endgroup$
    – Kaish
    Commented Dec 25, 2012 at 18:12

1 Answer 1

11
$\begingroup$

This isn't a stats question but a question relating to basic properties of calculus and algebra.

It may help to consider a simpler problem, to avoid any confusion about the issue:

$$\frac{\partial{}}{\partial{p_i}} \sum_{i = 1}^n p_i^2$$

Think of the summation written out:

$$ \frac{\partial{}}{\partial{p_i}} (p_1^2 + p_2^2 + ... + p_{i-1}^2 + p_i^2 + p_{i+1}^2 + ... + p_n^2) $$

Take the derivative term by term:

$$ = \frac{\partial{p_1^2}}{\partial{p_i}} + \frac{\partial{p_2^2}}{\partial{p_i}} + ... + \frac{\partial{p_{i-1}^2}}{\partial{p_i}} + \frac{\partial{p_{i}^2}}{\partial{p_i}} + \frac{\partial{p_{i+1}^2}}{\partial{p_i}} + ... + \frac{\partial{p_{n}^2}}{\partial{p_i}} $$

Now take those derivatives (leaving the $i^{\rm{th}}$ term unevaluated for the moment):

$$ = 0 + 0 + ... + 0 + \frac{\partial{p_{i}^2}}{\partial{p_i}} + 0 + ... + 0 $$

and we now see why the summation disappears - there's only one term that isn't zero:

$$ = \frac{\partial{p_{i}^2}}{\partial{p_i}} = 2p_i $$

Your question is the same but with a different, slightly more complicated function.

Regarding the original problem:

$$ l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$

is fine, but as soon as you took derivatives, you went astray.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.