6
$\begingroup$

$\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Suppose that independent random samples of sizes $n_1$ and $n_2$ are to be selected from normal populations with means $\mu_1$ and $\mu_2,$ respectively, and common variance $\sigma^2.$ For testing $H_0:\mu_1=\mu_2$ versus $H_a:\mu_1-\mu_2>0$ ($\sigma^2$ unknown), show that the likelihood ratio test reduces to the two-sample $t$ test presented in Section 10.8.

Note: This is Exercise 10.94 from Mathematical Statistics with Applications, 5th. Ed., by Wackerly, Mendenhall, and Scheaffer.

My Work So Far: We have the likelihood as $$L(\mu_1, \mu_2,\sigma^2)= \szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\sigma^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{1}{2\sigma^2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_1)^2 +\sum_{i=1}^{n_2}(y_i-\mu_2)^2}}.$$ To compute $L\big(\hat\Omega_0\big),$ we need to find the MLE for $\sigma^2:$ \begin{align*} \hat\sigma^2&=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_1)^2 +\sum_{i=1}^{n_2}(y_i-\mu_2)^2}. \end{align*} This is the MLE for $\sigma^2$ regardless of what $\mu_1$ and $\mu_2$ are. Thus, under $H_0,$ we have that $$\hat\sigma_0^2=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2},$$ and the unrestricted case is $$\hat\sigma^2=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}.$$ Under $H_0,\;\mu_1=\mu_2=\mu_0,$ so that \begin{align*} L\big(\hat\Omega_0\big) &=\szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\hat\sigma_0^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{n_1+n_2}{2}}\\ L\big(\hat\Omega\big) &=\szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\hat\sigma^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{n_1+n_2}{2}}, \end{align*} and the likelihood ratio is given by \begin{align*} \lambda &=\frac{L\big(\hat\Omega_0\big)}{L\big(\hat\Omega\big)}\\ &=\szdp{\frac{\hat\sigma^2}{\hat\sigma_0^2}}^{\!\!(n_1+n_2)/2}\\ &=\szdp{\frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2}}^{\!\!(n_1+n_2)/2}. \end{align*} It follows that the rejection region, $\lambda\le k,$ is equivalent to \begin{align*} \frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2}&<k^{2/(n_1+n_2)}=k'\\ \frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2+n_1(\overline{x}-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2+n_2(\overline{y}-\mu_0)^2}&<k'\\ \frac{1}{1+\dfrac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}}&<k'\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}&>\frac{1}{k'}-1=k''\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle(n_1-1)S_1^2+(n_2-1)S_2^2}&>k''\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\dfrac{(n_1-1)S_1^2+(n_2-1)S_2^2} {n_1+n_2-2}}&>k''(n_1+n_2-2)\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {S_p^2}&>k''(n_1+n_2-2). \end{align*} Here $$S_p^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}.$$

My Question: The goal is to get this expression somehow to look like $$t=\frac{\overline{x}-\overline{y}}{S_p\sqrt{1/n_1+1/n_2}}>t_{\alpha}.$$ But I don't see how I can convert my expression, with the same sign for $\overline{x}$ and $\overline{y},$ to the desired formula with its opposite signs. What am I missing?

$\endgroup$

1 Answer 1

7
+50
$\begingroup$

Under $H_0$, the MLEs are: $$\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}, \qquad \hat{\sigma}^2_0=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)$$ where $n=n_1+n_2$. Under $H_a$ we get: $$\hat{\mu}_1=\bar{X},~\hat{\mu}_2=\bar{Y},\qquad\hat{\sigma}^2=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} \right)$$ This is almost identical to your results.

Now, here is the trick (you can verify it easily): $$\sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2}=\sum_{i=1}^{n_1}{(X_i-\bar{X}+\bar{X}-\hat{\mu})^2}=\sum_{i=1}^{n_1}{(X_i-\bar{X})^2}+n_1(\bar{X}-\hat{\mu})^2$$ The same applies for the $Y_i$'s: $$\sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2}=\sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}+n_2(\bar{Y}-\hat{\mu})^2$$

Using $\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}$, we get that

$$\bar{X}-\hat{\mu}=\frac{n\bar{X}-n_1\bar{X}-n_2\bar{Y}}{n}=\frac{n_2}{n}(\bar{X}-\bar{Y})$$ and similarly $$\bar{Y}-\hat{\mu}=-\frac{n_1}{n}(\bar{X}-\bar{Y})$$

$$\hat{\sigma}_0^2=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} + n_1(\bar{X}-\hat{\mu})^2 + n_2(\bar{Y}-\hat{\mu})^2 \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} + \frac{n_1n_2^2}{n^2}(\bar{X}-\bar{Y})^2 + \frac{n_1^2n_2}{n^2}(\bar{X}-\bar{Y})^2 \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} + \frac{n_1n_2}{n}(\bar{X}-\bar{Y})^2 \right)$$

Now, the likelihood ratio is:

$$\lambda=\left(\frac{\hat{\sigma}^2}{\hat{\sigma}^2_0}\right)^{\frac{n}{2}}\le k,$$ which is equivalent to $$\left(\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2}\right)^{\frac{-n}{2}}\le k,$$ and this is equivalent to $$\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2} \le k'.$$

The last fraction is:

$$\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2}=\frac{\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} + \frac{n_1n_2}{n}(\bar{X}-\bar{Y})^2 \right)}{\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} \right)}$$ $$=1+\frac{n_1n_2}{n}\frac{(\bar{X}-\bar{Y})^2}{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}}.$$

Substracting $1$ from both sides and dividing by $\frac{n_1n_2}{n}$, the LRT is now

$$\frac{(\bar{X}-\bar{Y})^2}{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}} \le k''$$

This is equivalent to rejecting if

$$ \frac{\bar{X}-\bar{Y}}{\sqrt{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}}} \le k'''$$

which has the form of a $t$-test. $\blacksquare$

$\endgroup$
4
  • 1
    $\begingroup$ Thanks very much! Well-argued! $\endgroup$ Commented Sep 7, 2021 at 17:08
  • $\begingroup$ In the $t$ test, the pooled standard deviation has to be divided by $n-2$. In here, there is no provision as to what might be the divider. It could be $n-3$ or $n-1$. How can we derive the proper divider? $\endgroup$ Commented Aug 6, 2023 at 21:26
  • $\begingroup$ Did you mean to write $\bar X-\hat\mu=\frac{n\bar X-n_1\bar X-n_2\bar Y}n$ instead of $\bar X-\hat\mu=\bar X-\frac{n\bar X-n_1\bar X-n_2\bar Y}n$? Also, did you forget to write $\left(X-\bar X\right)^2$ and $(Y-\bar Y)^2$ instead of $(X-\bar X)$ and $(Y-\bar Y)$ until the end? $\endgroup$ Commented May 11 at 11:02
  • $\begingroup$ @PinkyWay indeed there were some mistakes, thank you $\endgroup$
    – Spätzle
    Commented May 15 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.