Asymptotic MLE Distribution With Two Random Samples

Question

I'm studiyng for an exam, and I found this problem which I can not managed to solve... I will be really grateful if someone can help me, thanks you.

Let $\left\{X_{1}, \ldots, X_{n}\right\} \sim^{iid} Exp(\lambda)$ and $\left\{Y_{1}, \ldots, Y_{m}\right\}\sim^{iid}Bern(p)$, whith $p=\exp(-\lambda)$, independent of each other.

Assume that the maximum likelihood estimator (MLE) $\widehat{\lambda}_{n, m}$ exists and $m=\alpha n$, where $\alpha$ is known. Using the Central Limit Theorem find the asymptotic distribution of the MLE where $n, m \rightarrow \infty$

Given the central limit theorem thing, I thought that the MLE is a function of the mean (then by using the delta method we have our distribution). But I don't see how the likelihood equation can be non-numerical solved, because we have the log-likelihood function

$$l(\lambda) =n\log(\lambda) -\lambda(\sum_{i=1}^{n}x_{i} +\sum_{i=1}^{m}y_{i}) +(m -\sum_{i=1}^{m}y_{i} ) \log( 1 -e^{-\lambda})$$

then

$$ \dfrac{\partial}{\partial\lambda}l(\lambda)=0:\quad \dfrac{n}{\lambda} -(\sum_{i=1}^{n}x_{i} +\sum_{i=1}^{m}y_{i}) +\dfrac{e^{-\lambda}(m -\sum_{i=1}^{m}y_{i})}{1 -e^{-\lambda}} =0$$

$$ \dfrac{n}{\lambda} -(\sum_{i=1}^{n}x_{i} +\sum_{i=1}^{m}y_{i}) +\dfrac{(m -\sum_{i=1}^{m}y_{i})}{e^{\lambda} -1} =0$$

Then, because of the "assume that the MLE exists" I thought of ussing the EMV asymptotic distribution (assuming the conditions), but I fail to see how to obtain de variance of the normal distribution because it suppose to be the fisher information for 1 observation, but we have two random samples with totally different distributon so I faced a dead end.

Answer 1

I would work with the two samples separately at first, since they are independent. The MLE of $\lambda$ from just the $X$s is $\tilde\lambda_n=1/\bar X_n$ and the MLE of $\lambda$ from just the $Y$s is $\check\lambda_m=-\log \bar Y_m$.

By the CLT and the delta method, $\tilde\lambda_n$ is asymptotically Normal, and you can write down its asymptotic mean and variance. Similarly, by the CLT and the delta method, $\check\lambda_n$ is asymptotically Normal, and you can write down its asymptotic mean and variance. These two are independent, so their joint asymptotic distribution is bivariate Normal and you know the means and variances.

Now, if you have two independent Normally-distributed estimates $(\tilde\lambda, \check\lambda)$ of the same thing, the optimal combination of them is a weighted average, weighted by the inverses of the variances. So that gets you an overall estimator with known asymptotic distribution.

If this were a real problem we'd be done, but it's an exam problem so we do need to worry whether this estimator is the MLE. It's asymptotically efficient (because the two components are MLEs and thus asymptotically efficient), and the MLE will be asymptotically efficient, so it must have the same asymptotic distribution as the MLE, and we are really done.

Now, I don't know that this is what the examiner had in mind. You can think of $X$ as first-event-times in independent Poisson processes and $Y$ as whether the first event happened before time 1, so there may be a clever argument that makes use of this connection and gives an explicit MLE or a more elegant link between the two sets of variables. But this is a derivation of the asymptotic distribution.

Answer 2

It's possible to solve this problem via "brute force" from just grinding through the calculus of the problem. To facilitate this analysis, let $\bar{x}_n \equiv \sum_{i=1}^n x_i/n$ and let $\bar{y}_m \equiv \sum_{i=1}^m y_i/m$ denote the sample means of the two groups. We can then write the log-likelihood function as:

$$\begin{align} \ell_{\mathbf{x}, \mathbf{y}}(\lambda) &= \sum_{i=1}^n \log \text{Exp}(x_i|\lambda) + \sum_{i=1}^m \log \text{Bern}(y_i|e^{-\lambda}) \\[6pt] &= \text{const} + n \log (\lambda) - n \bar{x}_n \lambda + m(1-\bar{y}_m) \log (1-e^{-\lambda}) - m \bar{y}_m \lambda \\[14pt] &= \text{const} - (n \bar{x}_n + m \bar{y}_m) \lambda + n \log (\lambda) + m(1-\bar{y}_m) \log (1-e^{-\lambda}). \\[6pt] \end{align}$$

We then have:

$$\begin{align} \frac{d \ell_{\mathbf{x}, \mathbf{y}}}{d \lambda}(\lambda) &= - (n \bar{x}_n + m \bar{y}_m) + \frac{n}{\lambda} + m(1-\bar{y}_m) \cdot \frac{e^{-\lambda}}{1-e^{-\lambda}}, \\[6pt] \frac{d^2 \ell_{\mathbf{x}, \mathbf{y}}}{d \lambda^2}(\lambda) &= - \frac{n}{\lambda^2} - m(1-\bar{y}_m) \cdot \frac{e^{-\lambda}}{(1-e^{-\lambda})^2}. \\[6pt] \end{align}$$

Since the second derivative is (almost surely) negative the log-likelihood function is strictly concave, so the MLE occurs at the unique critical point. The Fisher information is:

$$\begin{align} I(\lambda) &= \mathbb{E} \Bigg[ - \frac{d^2 \ell_{\mathbf{X}, \mathbf{Y}}}{d \lambda^2}(\lambda) \Bigg| \lambda \Bigg] \\[6pt] &= \mathbb{E} \Bigg[ \frac{n}{\lambda^2} + m(1-\bar{Y}_m) \cdot \frac{e^{-\lambda}}{(1-e^{-\lambda})^2} \Bigg| \lambda \Bigg] \\[6pt] &= \frac{n}{\lambda^2} + m(1-e^{-\lambda}) \cdot \frac{e^{-\lambda}}{(1-e^{-\lambda})^2} \\[6pt] &= \frac{n}{\lambda^2} + m \cdot \frac{e^{-\lambda}}{1-e^{-\lambda}}, \\[6pt] \end{align}$$

so its inverse is:

$$\begin{align} \frac{1}{I(\lambda)} = \frac{1}{n/\lambda^2 + m e^{-\lambda}/(1-e^{-\lambda})} = \frac{\lambda^2 (1-e^{-\lambda})}{n (1-e^{-\lambda}) + m \lambda^2 e^{-\lambda}}. \end{align}$$

Since the setup of your problem obeys the regularity conditions for asymptotic normality of the MLE, as $\min(n,m) \rightarrow \infty$ we have the asymptotic distribution:

$$\hat{\lambda}_{n,m} \overset{\text{Approx}}{\sim} \text{N} \Bigg( \lambda, \frac{\lambda^2 (1-e^{-\lambda})}{n (1-e^{-\lambda}) + m \lambda^2 e^{-\lambda}} \Bigg).$$

As $\min(n,m) \rightarrow \infty$ the variance converges to zero, so the MLE is a consistent estimator of $\lambda$.