0
$\begingroup$

This seem to silly but I wanted to confirm if the derivative of the log-likelihood $\hskip 2 pt l(x_i)$. The derivative of $$\frac{d (\sum_{i=1}^{M} log(x_i))}{dx} = \frac{1}{x_i} \sum_{i=1}^{M} \frac{1}{x_i}$$ Is this correct?

UPDATE of the Question : the pdf is $$p(x) = dP(x)/dx = d*x^{d-1}$$ where $P(x) = x^d$ $0<x<1$

Then the log-likelihood of x is $$ln L(x) = ln[p(x_1).p(x_2).\ldots.p(x_M)] = M*ln(d) + (d-1)\sum_{i=1}^{M} log(x_i)$$

Taking derivative of $ln(x)$ w.r.t to x or $x_i$ so as to find an estimate of $x$. The purpose is to find the minimum x's and the minimum value where found would correspond to the parameter. How do I proceed?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ shouldn't it be the sum of $1/x_i$? $\endgroup$
    – Drew75
    Commented Dec 3, 2013 at 20:36
  • $\begingroup$ yes, it should. The best way to see it is $\frac{d}{dx}\sum_i \log x_i=\sum_i \frac{d}{dx}\log x_i=\sum_i\frac{1}{x_i}$. $\endgroup$
    – fabee
    Commented Dec 3, 2013 at 20:41
  • $\begingroup$ @Drew75:Sorry, for the incomplete question, Ii have updated it. please have a look $\endgroup$
    – SKM
    Commented Dec 3, 2013 at 20:43
  • 1
    $\begingroup$ The derivative is zero, because the right hand side does not include any $x$, which is the variable with respect to which you are differentiating. If the $x_i$ are assumed to be functions of $x$, then this derivative is incorrect because it does not account for the $d(x_i)/dx.$ $\endgroup$
    – whuber ♦
    Commented Dec 3, 2013 at 21:36
  • 1
    $\begingroup$ The edit appears to confuse "$x$" with the parameter "$d$". As such it is nonsensical. Take a look at related questions on our site, such as stats.stackexchange.com/questions/32103, which illustrate the procedure generally, or stats.stackexchange.com/questions/4052, which examines a specific probability model. $\endgroup$
    – whuber ♦
    Commented Dec 3, 2013 at 22:26

1 Answer 1

Reset to default
2
$\begingroup$

If I understand correctly, the density is $$ f(x) = \begin{cases} d x^{d-1} & \text{if } 0 \le x \le 1 \\ 0 & \text{else} \end{cases}$$

The likelihood of $d$ is a function of the parameter $d$, the observations $x_1, \dots, x_n$ being considered as fixed: $$L(d) = \prod_{i=1}^n d x_i^{d-1} = d^n\prod_{i=1}^nx_i^{d-1} .$$ The log-likelihood is $\ell(d) = n \log d + (d-1) \sum_{i=1}^n \log(x_i)$.

It makes no sense to try to derive $\ell(d)$ with respect to the $x_i$. To find the MLE you have to derive it with respect to $d$: $$ {\partial \over \partial d} \ell(d) = n \times {1\over d} + \sum_{i=1}^n \log(x_i),$$ which leads to $$ \widehat{d} = - {n} \left( \sum_{i=1}^n \log(x_i) \right)^{-1}.$$

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you for your reply. Could you please let me know if the extension of the above is possible for this application : Considering xi to be the values of distances between 2 different signals - received signal X with actual parameters a,b & Y is the output of inverse filter. Distances are ri = ||Xi -Yi||.Instead of using the raw distances, using the pdf of the distances from above for both signals, how can I apply MLE on D=integration(f(xn|a,b))-f(yn|a1,b1))^2 dy, so as to maximize the probability of finding the D ? $\endgroup$
    – SKM
    Commented Dec 4, 2013 at 18:53
  • $\begingroup$ I am sorry, I don’t know anything about signal filtering and stuff. You should open a new question, describing in details your data, your problem and the rationale of your solution. $\endgroup$
    – Elvis
    Commented Dec 4, 2013 at 20:04
  • $\begingroup$ Thank you, but can you say from the pdf above, can I get an optimal value of x by applying MLE of Expectation Maximization? $\endgroup$
    – SKM
    Commented Dec 5, 2013 at 1:52
  • $\begingroup$ It seems like the estimator for $d$ is inverted in the final line. The estimator is actually for $1/d$. $\endgroup$
    – Drew75
    Commented Dec 5, 2013 at 7:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.