# Likelihood Ratio for two-sample Exponential distribution

Let $X$ and $Y$ be two independent random variables with respective pdfs:

$$f \left(x;\theta_i \right) =\begin{cases} \frac{1}{\theta_i} e^{-x/ {\theta_i}} \quad 0<x<\infty, 0<\theta_i< \infty \\ 0 \quad \text{elsewhere} \end{cases}$$

for $i=1,2$. Two indepedent samples are drawn in order to test $H_0: \theta_1 =\theta_2$ against $H_1 : \theta_1 \neq \theta_2$ of sizes $n_1$ and $n_2$ from these distributions. I need to show that the LRT $\Lambda$ can be written as a function of a statistic having $F$ distribution, under $H_0$.

Since the mle of this distribution is $\hat{\theta}=\bar{x}$, the LRT statistic becomes (I am skipping a few tedious steps here):

$$\Lambda =\frac{\bar{x}^{n_1} \bar{y}^{n_2} \left( n_1+n_2 \right)}{n_1 \bar{x}+n_2 \bar{y}}$$

I know that the $F$ distribution is defined as the quotient of two independent chi-square random variables, each one over their respective degrees of freedom. Additionally, since $X_i,Y_i \sim \Gamma \left( 1,\theta_1 \right)$ under the null, then $\sum X_i \sim \Gamma \left(n_1 ,\theta_1 \right)$ and$\sum Y_i \sim \Gamma \left(n_2, \theta_1 \right)$.

But how can I proceed from here? Any hints?

Thank you.

• Hint: An exponential random variable is linearly related to a $\chi^2$ random variable with two degrees of freedom, and thus a $\Gamma$ random variable with order parameter $n$ is linearly related to a $\chi^2$ random variable with $2n$ degrees of freedom. Commented Jan 3, 2014 at 18:44
• @DilipSarwate I can see that $Z= \frac {2}{\theta_1} \sum X_i \sim \chi^2 \left(2n_1 \right)$. Should I go on and try to reformulate my fraction according to that? Commented Jan 3, 2014 at 19:39
• Maybe you need to not skip the few tedious steps and actually derive the likelihood ratio from scratch instead of jumping to maximum-likelihood estimators. This is a problem about hypothesis testing not about maximum-likelihood estimation of an unknowm parameter $\theta_i$. Commented Jan 3, 2014 at 20:03
• @DilipSarwate You misunderstood. I have these intermediate steps written down but have not presented them here. This is what you get after simplification. Commented Jan 3, 2014 at 20:08
• Perhaps you can start by explaining to me (a non-statistician, by the way) what the T in LRT means. Commented Jan 3, 2014 at 23:29

If memory serves, it appears you have forgotten something in your LR statistic.

The likelihood function under the null is

$$L_{H_0} = \theta^{-n_1-n_2}\cdot \exp\left\{-\theta^{-1}\left(\sum x_i+\sum y_i\right)\right\}$$

and the MLE is

$$\hat \theta_0 = \frac {\sum x_i+\sum y_i}{n_1+n_2} = w_1\bar x +w_2 \bar y, \;\; w_1=\frac {n_1}{n_1+n_2},\;w_2=\frac {n_2}{n_1+n_2}$$

So$$L_{H_0}(\hat \theta_0) = (\hat \theta_0)^{-n_1-n_2}\cdot e^{-n_1-n_2}$$

Under the alternative, the likelihood is

$$L_{H_1} = \theta_1^{-n_1}\cdot \exp\left\{-\theta_1^{-1}\left(\sum x_i\right)\right\}\cdot \theta_2^{-n_2}\cdot \exp\left\{-\theta_2^{-1}\left(\sum y_i\right)\right\}$$

and the MLE's are

$$\hat \theta_1 = \frac {\sum x_i}{n_1} = \bar x, \qquad \hat \theta_2 = \frac {\sum y_i}{n_2} = \bar y$$

So $$L_{H_1}(\hat \theta_1,\,\hat \theta_2) = (\hat \theta_1)^{-n_1}(\hat \theta_2)^{-n_2}\cdot e^{-n_1-n_2}$$

Consider the ratio

$$\frac {L_{H_1}(\hat \theta_1,\,\hat \theta_2)}{L_{H_0}(\hat \theta_0)} = \frac {(\hat \theta_0)^{n_1+n_2}}{(\hat \theta_1)^{n_1}(\hat \theta_2)^{n_2}}=\left(\frac {\hat \theta_0}{\hat \theta_1}\right)^{n_1} \cdot \left(\frac {\hat \theta_0}{\hat \theta_2}\right)^{n_2}$$

$$= \left(w_1 + w_2 \frac {\bar y}{\bar x}\right)^{n_1} \cdot \left(w_1\frac {\bar x}{\bar y} + w_2 \right)^{n_2}$$

The sample means are independent -so I believe that you can now finish this.

• It's not very important but I think you should define the LRT as the reciprocal of the fraction you've used, see stats.ox.ac.uk/~dlunn/b8_02/b8pdf_8.pdf . Commented Jan 4, 2014 at 9:38
• The reciprocal was used because it helps with the algebraic manipulations. When this part is done, one just takes the negative of the outer powers. Commented Jan 4, 2014 at 10:50
• Alright. In order to show that the fraction $\frac{\bar{X}}{\bar{Y}}$ follows an F-distribution, it suffices to write it as $\frac{\frac{2\sum X_i}{2 \theta_1 n_1}}{\frac{2\sum Y_i}{2 \theta_1 n_2 }}$ , right? Commented Jan 4, 2014 at 11:40
• If that's a correct "link" between gammas and chi-squares, indeed. Commented Jan 4, 2014 at 11:56
• Yes, $\frac{2}{\theta_1} \sum X_i \sim \chi^2 \left( 2n_1 \right)$ And we also have to divide by the degrees of freedom, $2n_1$. Thank you very much. Commented Jan 4, 2014 at 11:59

The likelihood function given the sample $$\mathbf x=(x_1,\ldots,x_{n_1},y_1,\ldots,y_{n_2})$$ is given by

\begin{align} L(\theta_1,\theta_2)&=\frac{1}{\theta_1^{n_1}\theta_2^{n_2}}\,\exp\left[-\frac{1}{\theta_1}\sum_{i=1}^{n_1} x_i-\frac{1}{\theta_2}\sum_{i=1}^{n_2}y_i\right]\mathbf1_{\mathbf x>0},\quad\theta_1,\theta_2>0. \end{align}

The LR test criterion for testing $$H_0:\theta_1=\theta_2$$ against $$H_1:\theta_1\ne \theta_2$$ is of the form

\begin{align} \lambda(\mathbf x)&=\frac{\sup\limits_{\theta_1=\theta_2}L(\theta_1,\theta_2)}{\sup\limits_{\theta_1,\theta_2}L(\theta_1,\theta_2)} =\frac{L(\hat\theta,\hat\theta)}{L(\hat\theta_1,\hat\theta_2)}, \end{align} where $$\hat\theta$$ is the MLE of $$\theta_1=\theta_2$$ under $$H_0$$, and $$\hat\theta_i$$ is the unrestricted MLE of $$\theta_i$$ for $$i=1,2$$.

It is easily verified that $$\left(\hat\theta_1,\hat\theta_2\right)=(\bar x,\bar y)$$ and $$\hat\theta=\frac{n_1\bar x+n_2\bar y}{n_1+n_2}.$$

After some simplification we get this symmetry for the LRT criterion:

\begin{align} \lambda(\mathbf x)&=\underbrace{\text{constant}}_{>0}\left(\frac{n_1\bar x}{n_1\bar x+n_2\bar y}\right)^{\!\!n_1}\left(\frac{n_2\bar y}{n_1\bar x+n_2\bar y}\right)^{\!\!n_2} \\&=\text{constant}\cdot\,t^{n_1}(1-t_1)^{n_2},\quad\text{ where }t=\frac{n_1\bar x}{n_1\bar x+n_2\bar y} \\&=g(t),\,\text{say.} \end{align}

Studying the nature of the function $$g$$, we see that $$g'(t)\gtrless 0\;\iff\; t\lessgtr \frac{n_1}{n_1+n_2}.$$

Now since $$2n_1\overline X/\theta_1\sim \chi^2_{2n_1}$$ and $$2n_2\overline Y/\theta_2\sim \chi^2_{2n_2}$$ are independently distributed, we have $$\frac{\overline X}{\overline Y}\stackrel{H_0}{\sim}F_{2n_1,2n_2}.$$

Define $$v=\frac{n_1\overline x}{n_2\overline y},$$ so that $$t=\frac{v}{v+1}\quad\uparrow\, v$$

Therefore,

$$\lambda(\mathbf x)c_2,$$ where $$c_1,c_2$$ can be found from some size restriction and the fact that, under $$H_0$$, $$\frac{n_2}{n_1}\,v\sim F_{2n_1,2n_2}.$$