Let's say that I have a bunch of independent samples, $X_1, X_2, \dots, X_n$ and that they all follow Exponential($\theta_i$) distributions. (So they all have pdf $f(x_i)=\theta_i\exp(-\theta_iy_i)$.) I don't know if all the $\theta_i$s are equal or not, so I will assume the worst and say they are not for generalization purposes. How do I find the maximum likelihood estimate of this?

Here's my work so far:

$L = L(\theta_1, \theta_2, \dots, \theta_n | x_1, x_2, \dots, x_n)=\prod \theta_i \exp(-\sum \theta_iy_i)$

$\ln(L)=\sum\ln(\theta_i) - \sum\theta_ix_i$

$d/d\theta_i=\sum\frac{1}{\theta_i} - \sum x_i = 0$

$\sum \theta_i = \sum \frac{1}{x_i}$

Here's where I'm stuck - how can I say anything about a single $\theta_i$? Is $\hat{\theta}_i=\frac{1}{x_i}$??

  • $\begingroup$ Hint: Can you solve this problem for the case $n=1$? What is your solution? $\endgroup$ – whuber May 8 '15 at 16:45
  • $\begingroup$ In that case I get $\hat{\theta}=\frac{1}{x}$, as I had thought... But can I generalize that and say that $\theta_i=\frac{1}{x_i}$ without a proof...? $\endgroup$ – user76623 May 8 '15 at 17:22
  • $\begingroup$ Since the $X_i$ are independent, do you think it possible to improve on the estimates of the $\theta_i$? That is, what kind of information does $X_j$ give you about $\theta_i$ for $i\ne j$? To obtain rigor, any correct intuition you might have about the answer can be translated into properties of the likelihood function. (Incidentally, your formula for $d/d\theta_i$ is incorrect--and that might be confusing you. The correct expression has no summations over all the $i$.) $\endgroup$ – whuber May 8 '15 at 18:11
  • $\begingroup$ I'm afraid you lost me and I don't understand... How is the expression for $d/d\theta_i$ incorrect? If I have a single $\ln(\theta_i)$, then the derivative of that will be $1/\theta_i$. I have a sum of $\ln(\theta_i)$, so the derivative becomes $1/\sum\theta_i$. Along the same lines, if I have a single $\theta_i*x_i$ then the derivative will be $x_i$. Because it's a sum, the derivative is $\sum x_i$. I must be missing something fundamental here, because I can't find any alternative. $\endgroup$ – user76623 May 8 '15 at 19:05
  • $\begingroup$ You are not reading the notation correctly. To illustrate, suppose $n=2$. Then $\log(L) = (\log(\theta_1) +\log(\theta_2)) - (\theta_1x_1 + \theta_2x_2)$. When you differentiate with respect to $\theta_1$, all you get is $1/\theta_1 - x_1$: there is no sum because $\theta_1$ appears in one and only one term of each part. $\endgroup$ – whuber May 8 '15 at 19:27

$\ln(L)=\sum\ln(\theta_i) - \sum\theta_ix_i$

$\partial/\partial\theta_i=\frac{1}{\theta_i} - x_i = 0 $ $\implies$ $\hat\theta_i = \frac{1}{x_i}$

And $\partial^2/\partial^2\theta_i=\frac{-1}{\theta_i^2} < 0\ \forall\ \theta_i$ so $\hat\theta_i=\frac{1}{x_i}$ is indeed where the maxima occurs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.