Let $X_1,X_2,\ldots,X_N$ be i.i.d random variables having $\text{Exp}(1)$ distribution where $N$ is unknown. Suppose only $T=\max\{X_1,X_2,\ldots,X_N\}$ is observed.

I have to derive a most powerful test for testing $H_0:N=5$ versus $H_1:N=10$.

So $N$ is my parameter of interest. The joint distribution of $X_1,\ldots,X_N$ is of the form

$$f_N(x_1,\ldots,x_N)=\exp\left({-\sum_{i=1}^N x_i}\right)\mathbf1_{{x_1,\ldots,x_N>0}}$$

But this cannot be my likelihood function since $x_1,\ldots,x_N$ is not observed.

I am not sure how to express the above joint density as a function of $t$, the observed value of $T$. If I can write the joint density as some $f_N(t)$, then that would be my likelihood function given $t$.

I understand that the test is of the form $$\varphi(t)=\mathbf1_{\lambda(t)>k}$$

, where $\lambda$ is the likelihood ratio $$\lambda(t)=\frac{f_{H_1}(t)}{f_{H_0}(t)}$$

Any hint would be much appreciated.

As an aside to the actual question, I am curious if it is possible to find MLE of $N$.

  • $\begingroup$ Your joint distribution does not depend on the individual $x_i:$ it depends only on their sum, which is observed. $\endgroup$ – whuber Dec 21 '18 at 16:09
  • $\begingroup$ @whuber Yes it depends only on the sum. So $0<x_1,\ldots,x_N\le t\implies 0<\sum_{i=1}^N x_i\le Nt$. Does this imply $\sum_{i=1}^N x_i$ is observed? It only bounds the sum. $\endgroup$ – StubbornAtom Dec 21 '18 at 16:25
  • $\begingroup$ There's a missing step here: under your model assumptions, you can easily derive the distribution of $T.$ $\endgroup$ – whuber Dec 21 '18 at 16:49
  • 1
    $\begingroup$ @whuber Somehow did not realize that I am supposed to use the distribution of $T$. That makes it a standard problem. $\endgroup$ – StubbornAtom Dec 21 '18 at 17:28

Hint: the likelihood for $T$ is $$ f_{T}(t ; N) = N(1-e^{-t})^{N-1}e^{-t}. $$ To verify this, first find the cdf of $T$, and then differentiate. \begin{align*} F_T(t) &= P(T \le t)\\ &= [F_{X_i}(t)]^N. \end{align*}

Regarding your other question as to whether there is an MLE estimate for this: yes there is, but the likelihood is only defined on $\mathbb{N}^{+}$. This means that you cannot take a derivative and set it equal to zero.


If $N=5$ then $\Pr(X_1 \le x\ \&\ \cdots \ \&\ X_N\le x) = \left( 1-e^{-x} \right)^5,$ so you have a density function $$ \frac d {dx} \left( \left( 1-e^{-x} \right)^5 \right) = 5\left( 1 - e^{-x} \right)^4 \cdot e^{-x}. $$ And similarly if $N=10.$ So the likelihood function is $$ \begin{cases} L(5) = 5(1-e^{-x})^4 \cdot e^{-x} \\[8pt] L(10) = 10(1-e^{-x})^9 \cdot e^{-x} \end{cases} $$ where $x$ is the observed maximum value, and the ratio is $$ \frac{L(5)}{L(10)} = \frac 1 {2(1-e^{-x})^5}. $$ A small value of this ratio favors the alternative hypothesis $N=10.$ Equivalently, a large value of the observed maximum favors the alternative. Given the probability distribution of the maximum assuming the null hypothesis $N=5,$ you can find the critical value as a function of the level of the test.


As an aside to the actual question, I am curious if it is possible to find MLE of $N$.

This is quite straightforward, and simply requires you to derive the log-likelihood function and then use standard (discrete or continuous) calculus techniques to maximise. Using rules for the distribution of order statistics, the sampling distribution for $T$ is:

$$f_T(t) = N e^{-t} (1-e^{-t})^{N-1}.$$

The log-likelihood is:

$$\ell_t(N) = \ln N - t + (N-1)\ln(1-e^{-t}).$$

Now, maximising this function can be done either by using discrete calculus (i.e., using difference operators), or it can be done by treating $N$ as continuous and maximising using continuous calculus, and then discretising the result. For simplicity, we will do the latter. Taking $N$ as a real variable, the log-likelihood has corresponding score function and Fisher information (essentially the first and second derivatives, but with the sign reversed on the second) given by:

$$\begin{equation} \begin{aligned} s_t(N) &\equiv \frac{d \ell_t}{dN}(N) = \frac{1}{N} + \ln(1-e^{-t}), \\[12pt] I_t(N) &\equiv - \frac{d^2 \ell_t}{dN^2}(N) = \frac{1}{N^2} >0. \\[12pt] \end{aligned} \end{equation}$$

Since $I_t(N) > 0$ the likelihood function is a strictly concave function, so this means it has a unique MLE at its sole critical point $s_t(\hat{N}(t)) = 0$. This gives you the continuous MLE:

$$\hat{N}(t) = \frac{1}{\ln(1-e^{-t})}.$$

This will generally not be an integer value, so you obtain the corresponding discrete MLE by looking at the two integers either side of this value; the bigger one is the discrete MLE (which is almost surely unique).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.