Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have the log-likelihood function:

$$l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$

And I need to calculate the maximum likelihood estimator of $p_i$. When I do this, for some reason when differentiating, the summation sign vanishes. Why is this?

EDIT: To calculate the maximum likelihood estimators, I would differentiate my log likelihood, equal that to 0 and solve:

$$ \frac{\partial{l}}{\partial{p_i}} = \sum_{i = 1}^n \left( \frac{1}{p_i} - \frac{y_i}{1 - p_i} \right) = 0 $$

Rearranging gives me

$$ \sum \frac{1}{p_i} = \sum \left( \frac{y_i}{1 - p_i} \right) $$

Oh, does this now become

$$\frac{n}{p_i} = \frac{n y_i}{ 1 - pi} $$

And then I can divide through by n and rearrange to get my value for $\hat{p_i}$?

share|improve this question
    
It would help if you showed the steps you are taking in calculating this. –  Peter Flom Dec 25 '12 at 17:57
2  
Say that you take the derivative with respect to $p_1$ The derivative "can enter the summation", by linearity. Now, $p_1$ appears in the first term ($i = 1$) only. The derivative of all other terms ($i \neq 1$) are thus $0$. This is why the summation sign disappears. –  ocram Dec 25 '12 at 18:01
    
But in some of my other questions, I've had to leave the summation sign in to solve for my estimator. And would that not only give me the estimator for $p_1$ and not $p_i$? –  Kaish Dec 25 '12 at 18:05
2  
What I said is still valid for any $i$ between $1$ and $n$. $p_i$ appears in the $i$th term only; all other terms do not contribute to the derivative. Your first equation in your edit is thus wrong. –  ocram Dec 25 '12 at 18:08
    
So if I'm doing it for $i = i$, my first equation should read the same, just without the summation sign? –  Kaish Dec 25 '12 at 18:12

1 Answer 1

up vote 10 down vote accepted

This isn't a stats question but a question relating to basic properties of calculus and algebra.

It may help to consider a simpler problem, to avoid any confusion about the issue:

$$\frac{\partial{}}{\partial{p_i}} \sum_{i = 1}^n p_i^2$$

Think of the summation written out:

$$ \frac{\partial{}}{\partial{p_i}} (p_1^2 + p_2^2 + ... + p_{i-1}^2 + p_i^2 + p_{i+1}^2 + ... + p_n^2) $$

Take the derivative term by term:

$$ = \frac{\partial{p_1^2}}{\partial{p_i}} + \frac{\partial{p_2^2}}{\partial{p_i}} + ... + \frac{\partial{p_{i-1}^2}}{\partial{p_i}} + \frac{\partial{p_{i}^2}}{\partial{p_i}} + \frac{\partial{p_{i+1}^2}}{\partial{p_i}} + ... + \frac{\partial{p_{n}^2}}{\partial{p_i}} $$

Now take those derivatives (leaving the $i^{\rm{th}}$ term unevaluated for the moment):

$$ = 0 + 0 + ... + 0 + \frac{\partial{p_{i}^2}}{\partial{p_i}} + 0 + ... + 0 $$

and we now see why the summation disappears - there's only one term that isn't zero:

$$ = \frac{\partial{p_{i}^2}}{\partial{p_i}} = 2p_i $$

Your question is the same but with a different, slightly more complicated function.

Regarding the original problem:

$$ l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$

is fine, but as soon as you took derivatives, you went astray.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.