Expected Value for Sum of Squares in Reduced Model

Question

$\newcommand{\ol}[1]{\overline{#1}}\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$I am studying Design and Analysis of Experiments, 2nd Ed., by Dean, Voss, and Draguljic. In the context of a one-way ANOVA, Exercise 3.8 reads as follows:

Problem Statement: For the model in the previous exercise, find an unbiased estimator for $\sigma^2.$ (Hint: first calculate $E[ssE_0]$ in (3.5.10), p. 42.)

[AK Note]: The model is the reduced model for the one-way ANOVA: \begin{align*} Y_{it}&=\mu+\tau+\varepsilon_{it}^0,\quad i=1,\dots,\nu;\quad t=1,\dots,r_i,\\ &\varepsilon_{it}^0\sim N\left(0,\sigma^2\right)\;\text{independent RVs.} \end{align*}

My Work So Far: We take the hint. Eq. (3.5.10) states: $$ssE_0=\sum_i\sum_t\szdp{y_{it}-\ol{y}_{\bullet\bullet}}^{2}.$$ We define the sample variance for the $i$th treatment as $$S_i^2 =\sum_{t=1}^{r_i}\frac{\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2}}{r_i-1}$$ for this model. Then, in the random variable correspondence $SSE_0,$ we have $$SSE_0 =\sum_i\sum_t\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2} =\sum_i(r_i-1)S_i^2.$$ We compute $E\szdb{S_i^2}$: \begin{align*} E\szdb{S_i^2} &=E\szdb{\sum_{t=1}^{r_i}\frac{\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2}} {r_i-1}}\\ &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2}}} {r_i-1}\\ &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{\mu+\tau+\varepsilon^0_{it} -\ol{Y}_{\bullet\bullet}}^{\!2}}} {r_i-1}. \end{align*} But \begin{align*} \ol{Y}_{\bullet\bullet} &=\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}Y_{it}\\ &=\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}\szdp{\mu+\tau+\varepsilon_{it}^0}\\ &=\mu+\tau+\ol{\varepsilon}_{\bullet\bullet}^0, \end{align*} so that \begin{align*} E\szdb{S_i^2} &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{\varepsilon^0_{it} -\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}} {r_i-1}. \end{align*} Now then, we have \begin{align*} E\szdb{\szdp{\varepsilon_{it}^0-\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}} &=E\szdb{\szdp{\varepsilon_{it}^0}^{\!2}} -2E\szdb{\varepsilon_{it}^0\ol{\varepsilon}_{\bullet\bullet}^0} +E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}. \end{align*} Recall that, for any random variable $X,$ we have $$V(X)=E\szdp{X^2}-(E(X))^2,$$ so that \begin{align*} E\szdb{\szdp{\varepsilon_{it}^0}^{\!2}} &=V\szdp{\varepsilon_{it}^0} +\underbrace{E\szdb{\szdp{\varepsilon_{it}^0}}^2}_{=0}\\ &=\sigma^2,\\ E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}} &=V\szdp{\ol{\varepsilon}_{\bullet\bullet}^0} +\underbrace{E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}}^2}_{=0}\\ &=V\szdp{\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}\varepsilon_{it}^0}\\ &=\frac{1}{n^2}\sum_{i=1}^\nu\sum_{t=1}^{r_i}V(\varepsilon_{it}^0)\\ &=\frac{\sigma^2}{n},\\ E\szdb{\varepsilon_{js}^0\ol{\varepsilon}_{\bullet\bullet}^0} &=E\szdb{\varepsilon_{js}^0\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}\varepsilon_{it}^0}\\ &=\frac1n\, E\szdb{\varepsilon_{js}^0\sum_{i=1}^\nu\sum_{t=1}^{r_i}\varepsilon_{it}^0}\\ &=\frac1n\, \sum_{i=1}^\nu\sum_{t=1}^{r_i}E\szdb{\varepsilon_{js}^0\varepsilon_{it}^0}. \end{align*} In this last expression, if $j\not=i$ or $s\not= t,$ then the first random variable is independent of the second random variable, the expectation can be applied to each random variable separately, and the result is zero. It follows that only when $j=i$ and $s=t$ do we pick up anything non-zero: \begin{align*} E\szdb{\varepsilon_{js}^0\ol{\varepsilon}_{\bullet\bullet}^0} &=\frac1n\,E\szdb{\szdp{\varepsilon_{js}^0}^{\!2}}\\ &=\frac{\sigma^2}{n}. \end{align*} Plugging these expressions back into the previous computations reveals that \begin{align*} E\szdb{\szdp{\varepsilon_{it}^0-\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}} &=E\szdb{\szdp{\varepsilon_{it}^0}^{\!2}} -2E\szdb{\varepsilon_{it}^0\ol{\varepsilon}_{\bullet\bullet}^0} +E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}\\ &=\sigma^2-\frac{2\sigma^2}{n}+\frac{\sigma^2}{n}\\ &=\sigma^2-\frac{\sigma^2}{n}\\ &=\frac{(n-1)\sigma^2}{n}. \end{align*} In turn, we plug this back into the previous computations to reveal that \begin{align*} E\szdb{S_i^2} &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{\varepsilon^0_{it} -\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}} {r_i-1}\\ &=\sum_{t=1}^{r_i}\frac{(n-1)\sigma^2}{n(r_i-1)}\\ &=\frac{(n-1)\sigma^2}{n}\sum_{t=1}^{r_i}\frac{1}{r_i-1}\\ &=\frac{r_i(n-1)\sigma^2}{n(r_i-1)}. \end{align*} Finally, pulling ourselves all the way out of our computations, we have that \begin{align*} E\szdb{SSE_0} &=E\szdb{\sum_i(r_i-1)S_i^2}\\ &=\sum_i(r_i-1)E\szdb{S_i^2}\\ &=\sum_i(r_i-1)\frac{r_i(n-1)\sigma^2}{n(r_i-1)}\\ &=\sum_i\frac{r_i(n-1)\sigma^2}{n}\\ &=(n-1)\sigma^2. \end{align*} Hence, an unbiased estimator for $\sigma^2$ is $SSE_0/(n-1),$ as expected.

My Question: Is this correct? Perhaps more specifically: what justifies moving to the RV version: $ssE_0\to SSE_0?$

Answer 1

Denote $\alpha_i := \tau_i-\bar \tau_{\bullet}.$ Rewrite the One-Way ANOVA model as

\begin{align}Y_{it} &= \mu+\tau_i+\varepsilon_{it}^0\\ &= (\mu+ \bar \tau_{\bullet})+(\tau- \bar \tau_{\bullet})+\varepsilon_{it}\\ &= \mu^\prime + \alpha_i + \varepsilon_{it}\tag 1.\end{align}

It can be shown that

\begin{align}\mathbb E[\textrm{SST}] &= \sum r_i\alpha_i^2 + (v-1)\sigma^2,\tag 2\label 2\\ \mathbb E[\textrm{SSE}]&= (n-v)\sigma^2\tag 3\label 3.\end{align}

Under the null hypothesis, $\forall i, ~\alpha_i = 0;$ so $\eqref 2$ becomes

$$\mathbb E[\textrm{SST}] = (v-1)\sigma^2;\tag 4\label 4$$

adding $\eqref 4$ and $\eqref 3$ yields

\begin{align}\mathbb E[\textrm{SST}+ \textrm{SSE}] &= \mathbb E[\textrm{SSE}_0]\\ &= (n-1)\sigma^2.\tag 5\end{align}