# Likelihood Ratio for two-sample Exponential distribution

Let $X$ and $Y$ be two independent random variables with respective pdfs:

$$f \left(x;\theta_i \right) =\begin{cases} \frac{1}{\theta_i} e^{-x/ {\theta_i}} \quad 0<x<\infty, 0<\theta_i< \infty \\ 0 \quad \text{elsewhere} \end{cases}$$

for $i=1,2$. Two indepedent samples are drawn in order to test $H_0: \theta_1 =\theta_2$ against $H_1 : \theta_1 \neq \theta_2$ of sizes $n_1$ and $n_2$ from these distributions. I need to show that the LRT $\Lambda$ can be written as a function of a statistic having $F$ distribution, under $H_0$.

Since the mle of this distribution is $\hat{\theta}=\bar{x}$, the LRT statistic becomes (I am skipping a few tedious steps here):

$$\Lambda =\frac{\bar{x}^{n_1} \bar{y}^{n_2} \left( n_1+n_2 \right)}{n_1 \bar{x}+n_2 \bar{y}}$$

I know that the $F$ distribution is defined as the quotient of two independent chi-square random variables, each one over their respective degrees of freedom. Additionally, since $X_i,Y_i \sim \Gamma \left( 1,\theta_1 \right)$ under the null, then $\sum X_i \sim \Gamma \left(n_1 ,\theta_1 \right)$ and$\sum Y_i \sim \Gamma \left(n_2, \theta_1 \right)$.

But how can I proceed from here? Any hints?

Thank you.

• Hint: An exponential random variable is linearly related to a $\chi^2$ random variable with two degrees of freedom, and thus a $\Gamma$ random variable with order parameter $n$ is linearly related to a $\chi^2$ random variable with $2n$ degrees of freedom. Jan 3, 2014 at 18:44
• @DilipSarwate I can see that $Z= \frac {2}{\theta_1} \sum X_i \sim \chi^2 \left(2n_1 \right)$. Should I go on and try to reformulate my fraction according to that? Jan 3, 2014 at 19:39
• Maybe you need to not skip the few tedious steps and actually derive the likelihood ratio from scratch instead of jumping to maximum-likelihood estimators. This is a problem about hypothesis testing not about maximum-likelihood estimation of an unknowm parameter $\theta_i$. Jan 3, 2014 at 20:03
• @DilipSarwate You misunderstood. I have these intermediate steps written down but have not presented them here. This is what you get after simplification. Jan 3, 2014 at 20:08
• Perhaps you can start by explaining to me (a non-statistician, by the way) what the T in LRT means. Jan 3, 2014 at 23:29

If memory serves, it appears you have forgotten something in your LR statistic.

The likelihood function under the null is

$$L_{H_0} = \theta^{-n_1-n_2}\cdot \exp\left\{-\theta^{-1}\left(\sum x_i+\sum y_i\right)\right\}$$

and the MLE is

$$\hat \theta_0 = \frac {\sum x_i+\sum y_i}{n_1+n_2} = w_1\bar x +w_2 \bar y, \;\; w_1=\frac {n_1}{n_1+n_2},\;w_2=\frac {n_2}{n_1+n_2}$$

So$$L_{H_0}(\hat \theta_0) = (\hat \theta_0)^{-n_1-n_2}\cdot e^{-n_1-n_2}$$

Under the alternative, the likelihood is

$$L_{H_1} = \theta_1^{-n_1}\cdot \exp\left\{-\theta_1^{-1}\left(\sum x_i\right)\right\}\cdot \theta_2^{-n_2}\cdot \exp\left\{-\theta_2^{-1}\left(\sum y_i\right)\right\}$$

and the MLE's are

$$\hat \theta_1 = \frac {\sum x_i}{n_1} = \bar x, \qquad \hat \theta_2 = \frac {\sum y_i}{n_2} = \bar y$$

So $$L_{H_1}(\hat \theta_1,\,\hat \theta_2) = (\hat \theta_1)^{-n_1}(\hat \theta_2)^{-n_2}\cdot e^{-n_1-n_2}$$

Consider the ratio

$$\frac {L_{H_1}(\hat \theta_1,\,\hat \theta_2)}{L_{H_0}(\hat \theta_0)} = \frac {(\hat \theta_0)^{n_1+n_2}}{(\hat \theta_1)^{n_1}(\hat \theta_2)^{n_2}}=\left(\frac {\hat \theta_0}{\hat \theta_1}\right)^{n_1} \cdot \left(\frac {\hat \theta_0}{\hat \theta_2}\right)^{n_2}$$

$$= \left(w_1 + w_2 \frac {\bar y}{\bar x}\right)^{n_1} \cdot \left(w_1\frac {\bar x}{\bar y} + w_2 \right)^{n_2}$$

The sample means are independent -so I believe that you can now finish this.

• It's not very important but I think you should define the LRT as the reciprocal of the fraction you've used, see stats.ox.ac.uk/~dlunn/b8_02/b8pdf_8.pdf . Jan 4, 2014 at 9:38
• The reciprocal was used because it helps with the algebraic manipulations. When this part is done, one just takes the negative of the outer powers. Jan 4, 2014 at 10:50
• Alright. In order to show that the fraction $\frac{\bar{X}}{\bar{Y}}$ follows an F-distribution, it suffices to write it as $\frac{\frac{2\sum X_i}{2 \theta_1 n_1}}{\frac{2\sum Y_i}{2 \theta_1 n_2 }}$ , right? Jan 4, 2014 at 11:40
• If that's a correct "link" between gammas and chi-squares, indeed. Jan 4, 2014 at 11:56
• Yes, $\frac{2}{\theta_1} \sum X_i \sim \chi^2 \left( 2n_1 \right)$ And we also have to divide by the degrees of freedom, $2n_1$. Thank you very much. Jan 4, 2014 at 11:59

The likelihood function given the sample $$\mathbf x=(x_1,\ldots,x_{n_1},y_1,\ldots,y_{n_2})$$ is given by

\begin{align} L(\theta_1,\theta_2)&=\frac{1}{\theta_1^{n_1}\theta_2^{n_2}}\,\exp\left[-\frac{1}{\theta_1}\sum_{i=1}^{n_1} x_i-\frac{1}{\theta_2}\sum_{i=1}^{n_2}y_i\right]\mathbf1_{\mathbf x>0},\quad\theta_1,\theta_2>0. \end{align}

The LR test criterion for testing $$H_0:\theta_1=\theta_2$$ against $$H_1:\theta_1\ne \theta_2$$ is of the form

\begin{align} \lambda(\mathbf x)&=\frac{\sup\limits_{\theta_1=\theta_2}L(\theta_1,\theta_2)}{\sup\limits_{\theta_1,\theta_2}L(\theta_1,\theta_2)} =\frac{L(\hat\theta,\hat\theta)}{L(\hat\theta_1,\hat\theta_2)}, \end{align} where $$\hat\theta$$ is the MLE of $$\theta_1=\theta_2$$ under $$H_0$$, and $$\hat\theta_i$$ is the unrestricted MLE of $$\theta_i$$ for $$i=1,2$$.

It is easily verified that $$\left(\hat\theta_1,\hat\theta_2\right)=(\bar x,\bar y)$$ and $$\hat\theta=\frac{n_1\bar x+n_2\bar y}{n_1+n_2}.$$

After some simplification we get this symmetry for the LRT criterion:

\begin{align} \lambda(\mathbf x)&=\underbrace{\text{constant}}_{>0}\left(\frac{n_1\bar x}{n_1\bar x+n_2\bar y}\right)^{\!\!n_1}\left(\frac{n_2\bar y}{n_1\bar x+n_2\bar y}\right)^{\!\!n_2} \\&=\text{constant}\cdot\,t^{n_1}(1-t_1)^{n_2},\quad\text{ where }t=\frac{n_1\bar x}{n_1\bar x+n_2\bar y} \\&=g(t),\,\text{say.} \end{align}

Studying the nature of the function $$g$$, we see that $$g'(t)\gtrless 0\;\iff\; t\lessgtr \frac{n_1}{n_1+n_2}.$$

Now since $$2n_1\overline X/\theta_1\sim \chi^2_{2n_1}$$ and $$2n_2\overline Y/\theta_2\sim \chi^2_{2n_2}$$ are independently distributed, we have $$\frac{\overline X}{\overline Y}\stackrel{H_0}{\sim}F_{2n_1,2n_2}.$$

Define $$v=\frac{n_1\overline x}{n_2\overline y},$$ so that $$t=\frac{v}{v+1}\quad\uparrow\, v$$

Therefore,

$$\lambda(\mathbf x)c_2,$$ where $$c_1,c_2$$ can be found from some size restriction and the fact that, under $$H_0$$, $$\frac{n_2}{n_1}\,v\sim F_{2n_1,2n_2}.$$