5
$\begingroup$

$\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Suppose that independent random samples of sizes $n_1$ and $n_2$ are to be selected from normal populations with means $\mu_1$ and $\mu_2,$ respectively, and common variance $\sigma^2.$ For testing $H_0:\mu_1=\mu_2$ versus $H_a:\mu_1-\mu_2>0$ ($\sigma^2$ unknown), show that the likelihood ratio test reduces to the two-sample $t$ test presented in Section 10.8.

Note: This is Exercise 10.94 from Mathematical Statistics with Applications, 5th. Ed., by Wackerly, Mendenhall, and Scheaffer.

My Work So Far: We have the likelihood as $$L(\mu_1, \mu_2,\sigma^2)= \szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\sigma^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{1}{2\sigma^2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_1)^2 +\sum_{i=1}^{n_2}(y_i-\mu_2)^2}}.$$ To compute $L\big(\hat\Omega_0\big),$ we need to find the MLE for $\sigma^2:$ \begin{align*} \hat\sigma^2&=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_1)^2 +\sum_{i=1}^{n_2}(y_i-\mu_2)^2}. \end{align*} This is the MLE for $\sigma^2$ regardless of what $\mu_1$ and $\mu_2$ are. Thus, under $H_0,$ we have that $$\hat\sigma_0^2=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2},$$ and the unrestricted case is $$\hat\sigma^2=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}.$$ Under $H_0,\;\mu_1=\mu_2=\mu_0,$ so that \begin{align*} L\big(\hat\Omega_0\big) &=\szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\hat\sigma_0^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{n_1+n_2}{2}}\\ L\big(\hat\Omega\big) &=\szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\hat\sigma^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{n_1+n_2}{2}}, \end{align*} and the likelihood ratio is given by \begin{align*} \lambda &=\frac{L\big(\hat\Omega_0\big)}{L\big(\hat\Omega\big)}\\ &=\szdp{\frac{\hat\sigma^2}{\hat\sigma_0^2}}^{\!\!(n_1+n_2)/2}\\ &=\szdp{\frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2}}^{\!\!(n_1+n_2)/2}. \end{align*} It follows that the rejection region, $\lambda\le k,$ is equivalent to \begin{align*} \frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2}&<k^{2/(n_1+n_2)}=k'\\ \frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2+n_1(\overline{x}-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2+n_2(\overline{y}-\mu_0)^2}&<k'\\ \frac{1}{1+\dfrac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}}&<k'\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}&>\frac{1}{k'}-1=k''\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle(n_1-1)S_1^2+(n_2-1)S_2^2}&>k''\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\dfrac{(n_1-1)S_1^2+(n_2-1)S_2^2} {n_1+n_2-2}}&>k''(n_1+n_2-2)\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {S_p^2}&>k''(n_1+n_2-2). \end{align*} Here $$S_p^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}.$$

My Question: The goal is to get this expression somehow to look like $$t=\frac{\overline{x}-\overline{y}}{S_p\sqrt{1/n_1+1/n_2}}>t_{\alpha}.$$ But I don't see how I can convert my expression, with the same sign for $\overline{x}$ and $\overline{y},$ to the desired formula with its opposite signs. What am I missing?

$\endgroup$

1 Answer 1

6
+50
$\begingroup$

Under $H_0$, the MLEs are: $$\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}, \qquad \hat{\sigma}^2_0=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)$$ where $n=n_1+n_2$. Under $H_a$ we get: $$\hat{\mu}_1=\bar{X},~\hat{\mu}_2=\bar{Y},\qquad\hat{\sigma}^2=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} \right)$$ This is almost identical to your results.

Now, here is the trick (you can verify it easily): $$\sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2}=\sum_{i=1}^{n_1}{(X_i-\bar{X}+\bar{X}-\hat{\mu})^2}=\sum_{i=1}^{n_1}{(X_i-\bar{X})}+n_1(\bar{X}-\hat{\mu})^2$$ The same applies for the $Y_i$'s: $$\sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2}=\sum_{i=1}^{n_2}{(Y_i-\bar{Y})}+n_2(\bar{Y}-\hat{\mu})^2$$

Using $\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}$, we get that

$$\bar{X}-\hat{\mu}=\bar{X}-\frac{n\bar{X}-n_1\bar{X}-n_2\bar{Y}}{n}=\frac{n_2}{n}(\bar{X}-\bar{Y})$$ and similarly $$\bar{Y}-\hat{\mu}=-\frac{n_1}{n}(\bar{X}-\bar{Y})$$

$$\hat{\sigma}_0^2=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + n_1(\bar{X}-\hat{\mu})^2 + n_2(\bar{Y}-\hat{\mu})^2 \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + \frac{n_1n_2^2}{n^2}(\bar{X}-\bar{Y})^2 + \frac{n_1^2n_2}{n^2}(\bar{X}-\bar{Y})^2 \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + \frac{n_1n_2}{n}(\bar{X}-\bar{Y})^2 \right)$$

Now, the likelihood ratio is:

$$\lambda=\left(\frac{\hat{\sigma}^2}{\hat{\sigma}^2_0}\right)^{\frac{n}{2}}\le k,$$ which is equivalent to $$\left(\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2}\right)^{\frac{-n}{2}}\le k,$$ and this is equivalent to $$\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2} \le k'.$$

The last fraction is:

$$\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2}=\frac{\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + \frac{n_1n_2}{n}(\bar{X}-\bar{Y})^2 \right)}{\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} \right)}$$ $$=1+\frac{n_1n_2}{n}\frac{(\bar{X}-\bar{Y})^2}{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}}.$$

Substracting $1$ from both sides and dividing by $\frac{n_1n_2}{n}$, the LRT is now

$$\frac{(\bar{X}-\bar{Y})^2}{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}} \le k''$$

This id equivalent to rejecting if

$$ \frac{\bar{X}-\bar{Y}}{\sqrt{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}}} \le k'''$$

which has the form of a $t$-test. $\blacksquare$

$\endgroup$
2
  • 1
    $\begingroup$ Thanks very much! Well-argued! $\endgroup$ Sep 7, 2021 at 17:08
  • $\begingroup$ In the $t$ test, the pooled standard deviation has to be divided by $n-2$. In here, there is no provision as to what might be the divider. It could be $n-3$ or $n-1$. How can we derive the proper divider? $\endgroup$ Aug 6 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.