How can I show that the LRT of $H_0:\theta_0=\theta_1=...=\theta_k$ is given by the F test under one-way ANOVA assumptions?

Question

I am a bit curious as to how I can show this. I am aware that you can find this by using the union-intersection test, but it has been hinted that one can use the LRT test to find that LRT for $H_0$ is given by the F test.

my work:

I have that $L(\theta\mid y)=\prod^k_{i=1}\prod^{n_i}_{j=1}\frac{1}{\sqrt{2\pi\sigma^2}}\exp[-\frac{1}{2\sigma^2}(y_{ij}-\theta_i)^2]$.

Since $\sum^k_{i=1}\sum^{n_i}_{j=1}(y_{ij}-\theta_i)^2=\sum^k_{i=1}\sum^{n_i}_{j=1}(y_{ij}-\bar{y}_{i.})^2+\sum^k_{i=1}n_i(\bar{y}_{i.}-\theta_i)^2= SSW+\sum^k_{i=1}n_i(\bar{y}_{i.}-\theta_i)^2$

we get that $L(\theta\mid y)=(\frac{1}{2\pi\sigma^2})^{NK/2}\exp[-\frac{1}{2\sigma^2}(SSW+\sum^k_{i=1}n_i(\bar{y}_{i.}-\theta_i)^2)]$.

Now under $H_0$, I see that $\sum^k_{i=1}n_i(\bar{y}_{i.}-\theta_i)^2=\sum^k_{i=1}n_i(\bar{y}_{i.}-\bar{y}_{..})^2$.

However, I do not know where to go from here to show that the LRT of $H_0:\theta_1=\cdots=\theta_k$ is given by the F test.

Answer 1

I am going to state the known facts because the complete derivation is rather lengthy.

You have $y_{ij}\sim N(\theta_i,\sigma^2)$ independently for $i=1,\ldots,k;j=1,\ldots,n_i$ where $\sigma^2$ is unknown.

To test $$H_0:\theta_1=\theta_2=\cdots=\theta_k=\theta \text{ (say)}\quad\text{ vs. } \quad H_1:\text{not }H_0$$

Suppose $\boldsymbol\theta=(\theta_1,\theta_2,\ldots,\theta_k)$ and $\sum\limits_{i=1}^k n_i=n(>k)$.

Then likelihood function given the sample $\boldsymbol y=(y_{ij})_{i,j}$ is

$$L(\boldsymbol\theta,\sigma^2\mid \boldsymbol y)=\frac1{(\sigma\sqrt{2\pi})^n}\exp\left[-\frac1{2\sigma^2}\sum_{i,j}(y_{ij}-\theta_i)^2\right]$$

Unrestricted MLEs of $\boldsymbol\theta$ and $\sigma^2$ are

$$\widehat{\boldsymbol\theta}=(\overline y_1,\overline y_2,\ldots,\overline y_k)\quad,\quad \widehat{\sigma}^2=\frac{\text{SSE}}{n}$$

where $\overline y_{i\cdot}=\frac1{n_i}\sum_j y_{ij}$ is the $i$th group mean for all $i$ and $\text{SSE}=\sum_{i,j}(y_{ij}-\overline y_{i\cdot})^2$.

Restricted MLEs under $H_0$ are

$$\widehat{\widehat{\boldsymbol\theta}}=(\overline y,\overline y,\ldots,\overline y) \quad,\quad \widehat{\widehat{\sigma}}^2=\frac{\text{TSS}}{n}$$

where $\overline y=\frac1n\sum_i n_i\overline y_{i\cdot}$ is the grand mean and $$\text{TSS}=\sum_{i,j}(y_{ij}-\overline y)^2=\underbrace{\sum_i n_i(\overline y_{i\cdot}-\overline y)^2}_{\text{SSB}}+\underbrace{\sum_{i,j}(y_{ij}-\overline y_{i\cdot})^2}_{\text{SSE}}$$

The likelihood ratio simplifies as

$$ \Lambda(\boldsymbol y)=\frac{L\left(\widehat{\widehat{\boldsymbol\theta}}, \widehat{\widehat{\sigma}}^2\mid \boldsymbol y\right)}{L\left(\widehat{\boldsymbol\theta},\widehat{\sigma}^2\mid \boldsymbol y\right)} =\left(\frac{\widehat{\sigma}^2}{\widehat{\widehat{\sigma}}^2}\right)^{n/2} =\frac1{\left(\frac{\text{TSS}}{\text{SSE}}\right)^{n/2}} =\frac1{\left(1+\frac{\text{SSB}}{\text{SSE}}\right)^{n/2}} $$

We reject $H_0$ when $\Lambda(\boldsymbol y)<\text{constant}$, i.e. when $$\frac{\text{SSB}}{\text{SSE}}>\text{constant}$$

Now $\frac1{\sigma^2}\sum\limits_{j=1}^{n_i}(y_{ij}-\overline y_{i\cdot})^2 \sim \chi^2_{n_i-1}$ independently for all $i$, so that $$\frac{\text{SSE}}{\sigma^2}\sim \chi^2_{n-k}$$

Again $\overline y_{i\cdot}\stackrel{H_0}\sim N\left(\theta,\frac{\sigma^2}{n_i}\right)$ independently for all $i$, so as argued here,

$$\frac{\text{SSB}}{\sigma^2}\stackrel{H_0}\sim \chi^2_{k-1}$$

Moreover $\text{SSB}$ and $\text{SSE}$ are independent because $(y_{ij}-\overline y_{i\cdot},\overline y_{i\cdot}-\overline y)_{i,j}$ is jointly normal and

$$\operatorname{Cov}(y_{ij}-\overline y_{i\cdot},\overline y_{i\cdot}-\overline y)=0\quad,\forall\,i,j$$

These results are of course part of the Fisher-Cochran theorem.

Finally the test statistic is $$\frac{\text{SSB}/(k-1)}{\text{SSE}/(n-k)}\stackrel{H_0}\sim F_{k-1,n-k}$$

Answer 2

Plugging the MLEs into the expression for the multivariate normal density, the maximum likelihoods under each hypothesis $H_j$, $j=0,1$, is easily seen to be $$ \newcommand{\SSE}{\text{SSE}} (2\pi \SSE_j/n))^{-n/2}e^{-n/2}, $$ where $\SSE_j$, $j=0,1$, are the residual sums of squares. Hence, the ratio between the two maximum likelihoods is a strictly monotonic function of $$ \SSE_0/\SSE_1, $$ which in turn is a strictly monotonic function of $$ F=\frac{(\SSE_0-\SSE_1)/(p_1-p_0)}{\SSE_1/(n-p_1)}. $$ Hence the $F$-test is equivalent to the likelihood ratio test.