# Asymptotic MLE Distribution With Two Random Samples

I'm studiyng for an exam, and I found this problem which I can not managed to solve... I will be really grateful if someone can help me, thanks you.

Let $$\left\{X_{1}, \ldots, X_{n}\right\} \sim^{iid} Exp(\lambda)$$ and $$\left\{Y_{1}, \ldots, Y_{m}\right\}\sim^{iid}Bern(p)$$, whith $$p=\exp(-\lambda)$$, independent of each other.

Assume that the maximum likelihood estimator (MLE) $$\widehat{\lambda}_{n, m}$$ exists and $$m=\alpha n$$, where $$\alpha$$ is known. Using the Central Limit Theorem find the asymptotic distribution of the MLE where $$n, m \rightarrow \infty$$

Given the central limit theorem thing, I thought that the MLE is a function of the mean (then by using the delta method we have our distribution). But I don't see how the likelihood equation can be non-numerical solved, because we have the log-likelihood function

$$l(\lambda) =n\log(\lambda) -\lambda(\sum_{i=1}^{n}x_{i} +\sum_{i=1}^{m}y_{i}) +(m -\sum_{i=1}^{m}y_{i} ) \log( 1 -e^{-\lambda})$$

then

$$\dfrac{\partial}{\partial\lambda}l(\lambda)=0:\quad \dfrac{n}{\lambda} -(\sum_{i=1}^{n}x_{i} +\sum_{i=1}^{m}y_{i}) +\dfrac{e^{-\lambda}(m -\sum_{i=1}^{m}y_{i})}{1 -e^{-\lambda}} =0$$

$$\dfrac{n}{\lambda} -(\sum_{i=1}^{n}x_{i} +\sum_{i=1}^{m}y_{i}) +\dfrac{(m -\sum_{i=1}^{m}y_{i})}{e^{\lambda} -1} =0$$

Then, because of the "assume that the MLE exists" I thought of ussing the EMV asymptotic distribution (assuming the conditions), but I fail to see how to obtain de variance of the normal distribution because it suppose to be the fisher information for 1 observation, but we have two random samples with totally different distributon so I faced a dead end.

• An invitation to apply the CLT means you should expect the distribution to be approximately Normal and you will need to compute its parameters--most likely by computing its expectation and variance. That's where you should begin.
– whuber
Jan 17, 2022 at 22:19
• I'm sorry,I forgot to add that I initially thought that we can obtain a clossed form for the MLE (because they are telling us to use the CLT), but I don't manage to see how to solve the likelihood equation (see my edit). Then, because of the "assume that the MLE exists" I thought of ussing the EMV asymptotic distribution (assuming the conditions), but I fail to see how to obtain de variance of the normal distribution because it suppose to be the fisher information for 1 observation, but we have two random samples with tottally different distributon so I faced a dead end. Jan 18, 2022 at 1:02

I would work with the two samples separately at first, since they are independent. The MLE of $$\lambda$$ from just the $$X$$s is $$\tilde\lambda_n=1/\bar X_n$$ and the MLE of $$\lambda$$ from just the $$Y$$s is $$\check\lambda_m=-\log \bar Y_m$$.

By the CLT and the delta method, $$\tilde\lambda_n$$ is asymptotically Normal, and you can write down its asymptotic mean and variance. Similarly, by the CLT and the delta method, $$\check\lambda_n$$ is asymptotically Normal, and you can write down its asymptotic mean and variance. These two are independent, so their joint asymptotic distribution is bivariate Normal and you know the means and variances.

Now, if you have two independent Normally-distributed estimates $$(\tilde\lambda, \check\lambda)$$ of the same thing, the optimal combination of them is a weighted average, weighted by the inverses of the variances. So that gets you an overall estimator with known asymptotic distribution.

If this were a real problem we'd be done, but it's an exam problem so we do need to worry whether this estimator is the MLE. It's asymptotically efficient (because the two components are MLEs and thus asymptotically efficient), and the MLE will be asymptotically efficient, so it must have the same asymptotic distribution as the MLE, and we are really done.

Now, I don't know that this is what the examiner had in mind. You can think of $$X$$ as first-event-times in independent Poisson processes and $$Y$$ as whether the first event happened before time 1, so there may be a clever argument that makes use of this connection and gives an explicit MLE or a more elegant link between the two sets of variables. But this is a derivation of the asymptotic distribution.

• That's makes sense... I don't know neither what the examiner had in mind, because this is a test from 2-3 years ago. But as I said, this totally makes sense. Thanks you very much!!! Jan 18, 2022 at 2:51

It's possible to solve this problem via "brute force" from just grinding through the calculus of the problem. To facilitate this analysis, let $$\bar{x}_n \equiv \sum_{i=1}^n x_i/n$$ and let $$\bar{y}_m \equiv \sum_{i=1}^m y_i/m$$ denote the sample means of the two groups. We can then write the log-likelihood function as:

\begin{align} \ell_{\mathbf{x}, \mathbf{y}}(\lambda) &= \sum_{i=1}^n \log \text{Exp}(x_i|\lambda) + \sum_{i=1}^m \log \text{Bern}(y_i|e^{-\lambda}) \\[6pt] &= \text{const} + n \log (\lambda) - n \bar{x}_n \lambda + m(1-\bar{y}_m) \log (1-e^{-\lambda}) - m \bar{y}_m \lambda \\[14pt] &= \text{const} - (n \bar{x}_n + m \bar{y}_m) \lambda + n \log (\lambda) + m(1-\bar{y}_m) \log (1-e^{-\lambda}). \\[6pt] \end{align}

We then have:

\begin{align} \frac{d \ell_{\mathbf{x}, \mathbf{y}}}{d \lambda}(\lambda) &= - (n \bar{x}_n + m \bar{y}_m) + \frac{n}{\lambda} + m(1-\bar{y}_m) \cdot \frac{e^{-\lambda}}{1-e^{-\lambda}}, \\[6pt] \frac{d^2 \ell_{\mathbf{x}, \mathbf{y}}}{d \lambda^2}(\lambda) &= - \frac{n}{\lambda^2} - m(1-\bar{y}_m) \cdot \frac{e^{-\lambda}}{(1-e^{-\lambda})^2}. \\[6pt] \end{align}

Since the second derivative is (almost surely) negative the log-likelihood function is strictly concave, so the MLE occurs at the unique critical point. The Fisher information is:

\begin{align} I(\lambda) &= \mathbb{E} \Bigg[ - \frac{d^2 \ell_{\mathbf{X}, \mathbf{Y}}}{d \lambda^2}(\lambda) \Bigg| \lambda \Bigg] \\[6pt] &= \mathbb{E} \Bigg[ \frac{n}{\lambda^2} + m(1-\bar{Y}_m) \cdot \frac{e^{-\lambda}}{(1-e^{-\lambda})^2} \Bigg| \lambda \Bigg] \\[6pt] &= \frac{n}{\lambda^2} + m(1-e^{-\lambda}) \cdot \frac{e^{-\lambda}}{(1-e^{-\lambda})^2} \\[6pt] &= \frac{n}{\lambda^2} + m \cdot \frac{e^{-\lambda}}{1-e^{-\lambda}}, \\[6pt] \end{align}

so its inverse is:

\begin{align} \frac{1}{I(\lambda)} = \frac{1}{n/\lambda^2 + m e^{-\lambda}/(1-e^{-\lambda})} = \frac{\lambda^2 (1-e^{-\lambda})}{n (1-e^{-\lambda}) + m \lambda^2 e^{-\lambda}}. \end{align}

Since the setup of your problem obeys the regularity conditions for asymptotic normality of the MLE, as $$\min(n,m) \rightarrow \infty$$ we have the asymptotic distribution:

$$\hat{\lambda}_{n,m} \overset{\text{Approx}}{\sim} \text{N} \Bigg( \lambda, \frac{\lambda^2 (1-e^{-\lambda})}{n (1-e^{-\lambda}) + m \lambda^2 e^{-\lambda}} \Bigg).$$

As $$\min(n,m) \rightarrow \infty$$ the variance converges to zero, so the MLE is a consistent estimator of $$\lambda$$.

• Thanks for answering! About your last result I have a question... It will be correct to have n and m into the distribution parameters considering that we are tending m and n to infinity? Jan 19, 2022 at 19:04
• The asymptotic result means that if $m$ and $n$ are large then the approximating distribution shown here will be a close approximation to the true distribution of the MLE. I have not stated the formal limit result, but it is simple to do by rearranging the terms.
– Ben
Jan 19, 2022 at 22:00
• I see... Thanks you! Jan 20, 2022 at 21:44