4
$\begingroup$

Let's say that I have a bunch of independent samples, $X_1, X_2, \dots, X_n$ and that they all follow Exponential($\theta_i$) distributions. (So they all have pdf $f(x_i)=\theta_i\exp(-\theta_iy_i)$.) I don't know if all the $\theta_i$s are equal or not, so I will assume the worst and say they are not for generalization purposes. How do I find the maximum likelihood estimate of this?

Here's my work so far:

$L = L(\theta_1, \theta_2, \dots, \theta_n | x_1, x_2, \dots, x_n)=\prod \theta_i \exp(-\sum \theta_iy_i)$

$\ln(L)=\sum\ln(\theta_i) - \sum\theta_ix_i$

$d/d\theta_i=\sum\frac{1}{\theta_i} - \sum x_i = 0$

$\sum \theta_i = \sum \frac{1}{x_i}$

Here's where I'm stuck - how can I say anything about a single $\theta_i$? Is $\hat{\theta}_i=\frac{1}{x_i}$??

Improve this question
$\endgroup$
6
  • $\begingroup$ Hint: Can you solve this problem for the case $n=1$? What is your solution? $\endgroup$
    – whuber ♦
    May 8, 2015 at 16:45
  • $\begingroup$ In that case I get $\hat{\theta}=\frac{1}{x}$, as I had thought... But can I generalize that and say that $\theta_i=\frac{1}{x_i}$ without a proof...? $\endgroup$
    – user76623
    May 8, 2015 at 17:22
  • $\begingroup$ Since the $X_i$ are independent, do you think it possible to improve on the estimates of the $\theta_i$? That is, what kind of information does $X_j$ give you about $\theta_i$ for $i\ne j$? To obtain rigor, any correct intuition you might have about the answer can be translated into properties of the likelihood function. (Incidentally, your formula for $d/d\theta_i$ is incorrect--and that might be confusing you. The correct expression has no summations over all the $i$.) $\endgroup$
    – whuber ♦
    May 8, 2015 at 18:11
  • $\begingroup$ I'm afraid you lost me and I don't understand... How is the expression for $d/d\theta_i$ incorrect? If I have a single $\ln(\theta_i)$, then the derivative of that will be $1/\theta_i$. I have a sum of $\ln(\theta_i)$, so the derivative becomes $1/\sum\theta_i$. Along the same lines, if I have a single $\theta_i*x_i$ then the derivative will be $x_i$. Because it's a sum, the derivative is $\sum x_i$. I must be missing something fundamental here, because I can't find any alternative. $\endgroup$
    – user76623
    May 8, 2015 at 19:05
  • $\begingroup$ You are not reading the notation correctly. To illustrate, suppose $n=2$. Then $\log(L) = (\log(\theta_1) +\log(\theta_2)) - (\theta_1x_1 + \theta_2x_2)$. When you differentiate with respect to $\theta_1$, all you get is $1/\theta_1 - x_1$: there is no sum because $\theta_1$ appears in one and only one term of each part. $\endgroup$
    – whuber ♦
    May 8, 2015 at 19:27

1 Answer 1

Reset to default
2
$\begingroup$

$\ln(L)=\sum\ln(\theta_i) - \sum\theta_ix_i$

$\partial/\partial\theta_i=\frac{1}{\theta_i} - x_i = 0 $ $\implies$ $\hat\theta_i = \frac{1}{x_i}$

And $\partial^2/\partial^2\theta_i=\frac{-1}{\theta_i^2} < 0\ \forall\ \theta_i$ so $\hat\theta_i=\frac{1}{x_i}$ is indeed where the maxima occurs.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.