9
$\begingroup$

I am stuck on how to solve this problem.

So, we have two sequences of random variables, $X_i$ and $Y_i$ for $i=1,...,n$. Now, $X$ and $Y$ are independent exponential distributions with parameters $\lambda$ and $\mu$. However, instead of observing $X$ and $Y$, we observe instead $Z$ and $W$.

$Z=\min(X_i,Y_i)$ and $W=1$ if $Z_i=X_i$ and 0 if $Z_i=Y_i$. I have to find closed-forms for the maximum likelihood estimators of $\lambda$ and $\mu$ on the basis of $Z$ and $W$. Further, we need to show that these are global maxima.

Now, I know that the minimum of two independent exponentials is itself exponential, with the rate equal to the sum of rates, so we know that $Z$ is exponential with parameter $\lambda+\mu$. Thus our maximum likelihood estimator is: $\hat{\lambda}+\hat{\mu}=\bar{Z}$.

But I'm stuck with where to go from here. I know that $W$ is a Bernoulli distribution with parameter $p=P(Z_i=X_i)$, but I don't know how to go about converting this into a statement about one of the parameters. For example, what would the MLE $\bar{W}$ be estimating in terms of $\lambda$ and/or $\mu$? I understand that if $Z_i=X_i$, then $\mu=0$, but I'm having a hard time figuring out how to come up with any algebraic statement, here.

UPDATE 1: So I have been told in the comments to derive the likelihood for the joint distribution of $Z$ and $W$.

So $f(Z,W)=f(Z|W=1)\cdot p+f(Z|W=0)\cdot (1-p)$ where $p=P(Z_i=X_i)$. Correct? I don't know how else to derive a joint distribution in this case, since $Z$ and $W$ are not independent.

So this gives us, $f(Z_i,W_i)=p\lambda e^{-\lambda z_i}+(1-p)\mu e^{-\mu z_i}$, by the definition of $W$ above. But now what? This doesn't get me anywhere. If I go through the steps of calculating the likelihood, I get: (using $m$ and $n$ as the sample sizes for each part of the mixture...)

$L(\lambda,\mu)=p^m\lambda^m e^{-\lambda \sum{z_i}}+(1-p)^n\mu^n e^{-\mu \sum{z_i}}$

$\log L=m\log p+m\log\lambda-\lambda \sum{z_i}+n\log(1-p)+n\log\mu-\mu \sum{z_i}$

If I take the partial derivatives, this tells me that my MLE estimates for $\lambda$ and $\mu$ are just the average of the $Z$'s conditional on $W$. That is,

$\hat{\lambda}=\frac{\sum{Z_i}}{m}$

$\hat{\mu}=\frac{\sum{Z_i}}{n}$

and

$\hat{p}=\frac{m}{n+m}$

$\endgroup$
  • 1
    $\begingroup$ Having just answered a similar MLE question today, may I direct you towards that solution for some ideas? The relationship between the questions is that your data also break up naturally into two disjoint groups: those where $W=0$ and those where $W=1$. It all comes down to writing down the likelihood for an observation of the form $(Z,W)=(z,0)$; the symmetry between $X$ and $Y$, $\mu$ and $\lambda$, immediately produces the likelihood for data of the form $(z,1)$ and then you're off and running. $\endgroup$ – whuber Nov 17 '14 at 20:56
  • $\begingroup$ Do not rush to writing the maximum likelihood! First, express the joint distribution of $(Z,W)$, then deduce the likelihood associated with the sample of $(Z_i,W)=_i)$, which happens to be closed-form thanks to the exponential assumption. Then and only then you can try to maximise the function and hence derive the maximum likelihood. $\endgroup$ – Xi'an Nov 17 '14 at 20:57
  • $\begingroup$ @whuber: (+1) it is rather straightforward indeed and involves the separation between the $(z_i,1)$'s and the $(z_i,0)$ but both groups involve both $\mu$ and $\lambda$, since they bring information on both $X_i$ and $Y_i$, since $W_i=\mathbb{I}(X_i<Y_i)$. $\endgroup$ – Xi'an Nov 17 '14 at 21:00
  • 2
    $\begingroup$ @Xi'an That's right--and the parallels with the Normal-theory example I link to continue to hold, because there both groups provide information about the common parameter $\sigma$ (the scale), whose estimate will thereby involve "pooling" data from the groups. Here it will be seen that $\bar W$ tells us how the estimate of $\lambda+\mu$ (the rate, or inverse scale, for $Z$) should be apportioned into separate estimates of $\lambda$ and $\mu$. $\endgroup$ – whuber Nov 17 '14 at 21:05
  • $\begingroup$ I've read through the other thread, whuber, but I honestly don't understand how to apply that to this example. Z and W aren't independent, so how do I derive the joint distribution? $\endgroup$ – Ryan Simmons Nov 17 '14 at 22:04
1
$\begingroup$

I don't have enough points to comment, so I will write here. I think the problem you post can be viewed from a survival analysis perspective, if you consider the following:

$X_i$: True survival time,

$Y_i$: Censoring time,

Both have an exponential distribution with $X$ and $Y$ independent. Then $Z_i$ is the observed survival time and $W_i$ the censoring indicator.

If you are familiar with survival analysis, I believe you can start from this point.

Notes: A good source: Analysis of Survival Data by D.R.Cox and D.Oakes

Below is an example: Assuming the p.d.f of the survival time distribution is $f(t)=\rho e^{-\rho t}$. Then the survival function is: $S(t)=e^{-\rho t}$. And the log-likelihood is:

$\mathcal{l}= \sum_u \log f(z_i) + \sum_c \log S(z_i)$

with summation over uncensored people ($u$) and censored people ($c$) respectively.

Due to the fact that $f(t)=h(t)S(t)$ where h(t) is the hazard function, this can be written:

$\mathcal{l}= \sum_u \log h(z_i) + \sum \log S(z_i)$

$\mathcal{l}= \sum_u \log \rho - \rho \sum z_i$

And the maximum likelihood estimator $\hat{\rho}$ of $\rho$ is:

$\hat{\rho}=d/\sum z_i$ where $d$ is the total number of cases of $W_i=1$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.