# Expected Value for Sum of Squares in Reduced Model

$$\newcommand{\ol}[1]{\overline{#1}}\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$$I am studying Design and Analysis of Experiments, 2nd Ed., by Dean, Voss, and Draguljic. In the context of a one-way ANOVA, Exercise 3.8 reads as follows:

Problem Statement: For the model in the previous exercise, find an unbiased estimator for $$\sigma^2.$$ (Hint: first calculate $$E[ssE_0]$$ in (3.5.10), p. 42.)

[AK Note]: The model is the reduced model for the one-way ANOVA: \begin{align*} Y_{it}&=\mu+\tau+\varepsilon_{it}^0,\quad i=1,\dots,\nu;\quad t=1,\dots,r_i,\\ &\varepsilon_{it}^0\sim N\left(0,\sigma^2\right)\;\text{independent RVs.} \end{align*}

My Work So Far: We take the hint. Eq. (3.5.10) states: $$ssE_0=\sum_i\sum_t\szdp{y_{it}-\ol{y}_{\bullet\bullet}}^{2}.$$ We define the sample variance for the $$i$$th treatment as $$S_i^2 =\sum_{t=1}^{r_i}\frac{\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2}}{r_i-1}$$ for this model. Then, in the random variable correspondence $$SSE_0,$$ we have $$SSE_0 =\sum_i\sum_t\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2} =\sum_i(r_i-1)S_i^2.$$ We compute $$E\szdb{S_i^2}$$: \begin{align*} E\szdb{S_i^2} &=E\szdb{\sum_{t=1}^{r_i}\frac{\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2}} {r_i-1}}\\ &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{Y_{it}-\ol{Y}_{\bullet\bullet}}^{\!2}}} {r_i-1}\\ &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{\mu+\tau+\varepsilon^0_{it} -\ol{Y}_{\bullet\bullet}}^{\!2}}} {r_i-1}. \end{align*} But \begin{align*} \ol{Y}_{\bullet\bullet} &=\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}Y_{it}\\ &=\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}\szdp{\mu+\tau+\varepsilon_{it}^0}\\ &=\mu+\tau+\ol{\varepsilon}_{\bullet\bullet}^0, \end{align*} so that \begin{align*} E\szdb{S_i^2} &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{\varepsilon^0_{it} -\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}} {r_i-1}. \end{align*} Now then, we have \begin{align*} E\szdb{\szdp{\varepsilon_{it}^0-\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}} &=E\szdb{\szdp{\varepsilon_{it}^0}^{\!2}} -2E\szdb{\varepsilon_{it}^0\ol{\varepsilon}_{\bullet\bullet}^0} +E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}. \end{align*} Recall that, for any random variable $$X,$$ we have $$V(X)=E\szdp{X^2}-(E(X))^2,$$ so that \begin{align*} E\szdb{\szdp{\varepsilon_{it}^0}^{\!2}} &=V\szdp{\varepsilon_{it}^0} +\underbrace{E\szdb{\szdp{\varepsilon_{it}^0}}^2}_{=0}\\ &=\sigma^2,\\ E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}} &=V\szdp{\ol{\varepsilon}_{\bullet\bullet}^0} +\underbrace{E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}}^2}_{=0}\\ &=V\szdp{\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}\varepsilon_{it}^0}\\ &=\frac{1}{n^2}\sum_{i=1}^\nu\sum_{t=1}^{r_i}V(\varepsilon_{it}^0)\\ &=\frac{\sigma^2}{n},\\ E\szdb{\varepsilon_{js}^0\ol{\varepsilon}_{\bullet\bullet}^0} &=E\szdb{\varepsilon_{js}^0\frac1n\sum_{i=1}^\nu\sum_{t=1}^{r_i}\varepsilon_{it}^0}\\ &=\frac1n\, E\szdb{\varepsilon_{js}^0\sum_{i=1}^\nu\sum_{t=1}^{r_i}\varepsilon_{it}^0}\\ &=\frac1n\, \sum_{i=1}^\nu\sum_{t=1}^{r_i}E\szdb{\varepsilon_{js}^0\varepsilon_{it}^0}. \end{align*} In this last expression, if $$j\not=i$$ or $$s\not= t,$$ then the first random variable is independent of the second random variable, the expectation can be applied to each random variable separately, and the result is zero. It follows that only when $$j=i$$ and $$s=t$$ do we pick up anything non-zero: \begin{align*} E\szdb{\varepsilon_{js}^0\ol{\varepsilon}_{\bullet\bullet}^0} &=\frac1n\,E\szdb{\szdp{\varepsilon_{js}^0}^{\!2}}\\ &=\frac{\sigma^2}{n}. \end{align*} Plugging these expressions back into the previous computations reveals that \begin{align*} E\szdb{\szdp{\varepsilon_{it}^0-\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}} &=E\szdb{\szdp{\varepsilon_{it}^0}^{\!2}} -2E\szdb{\varepsilon_{it}^0\ol{\varepsilon}_{\bullet\bullet}^0} +E\szdb{\szdp{\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}\\ &=\sigma^2-\frac{2\sigma^2}{n}+\frac{\sigma^2}{n}\\ &=\sigma^2-\frac{\sigma^2}{n}\\ &=\frac{(n-1)\sigma^2}{n}. \end{align*} In turn, we plug this back into the previous computations to reveal that \begin{align*} E\szdb{S_i^2} &=\sum_{t=1}^{r_i}\frac{E\szdb{\szdp{\varepsilon^0_{it} -\ol{\varepsilon}_{\bullet\bullet}^0}^{\!2}}} {r_i-1}\\ &=\sum_{t=1}^{r_i}\frac{(n-1)\sigma^2}{n(r_i-1)}\\ &=\frac{(n-1)\sigma^2}{n}\sum_{t=1}^{r_i}\frac{1}{r_i-1}\\ &=\frac{r_i(n-1)\sigma^2}{n(r_i-1)}. \end{align*} Finally, pulling ourselves all the way out of our computations, we have that \begin{align*} E\szdb{SSE_0} &=E\szdb{\sum_i(r_i-1)S_i^2}\\ &=\sum_i(r_i-1)E\szdb{S_i^2}\\ &=\sum_i(r_i-1)\frac{r_i(n-1)\sigma^2}{n(r_i-1)}\\ &=\sum_i\frac{r_i(n-1)\sigma^2}{n}\\ &=(n-1)\sigma^2. \end{align*} Hence, an unbiased estimator for $$\sigma^2$$ is $$SSE_0/(n-1),$$ as expected.

My Question: Is this correct? Perhaps more specifically: what justifies moving to the RV version: $$ssE_0\to SSE_0?$$

• The calculation seems legit at a quick glance. One thing that is bugging me is how $E[ssE_0]$ is possible: isn't $ssE_0$ the observed variable and $SSE_0$ the random variable, that is won't $E[SSE_0]$ be taken? Commented Dec 5, 2022 at 23:55
• I'm not sure I understand your comment when you say "won't $E[SSE_0]$ be taken?" It is true, I believe that $ssE_0$ is the observed value of the sum of squares, whereas $SSE_0$ is the corresponding random variable. And that's the main thrust of my question: why can we just switch from the one to the other? Commented Dec 6, 2022 at 2:28
• What I am saying is that whether it would be right to take expectation of observed value. Say, I have a random variable $X;$ now $EX$ makes sense but does the expectation of observed value of $X,$ say $x,$ i.e. $Ex$ make sense? Commented Dec 6, 2022 at 2:37

Denote $$\alpha_i := \tau_i-\bar \tau_{\bullet}.$$ Rewrite the One-Way ANOVA model as

\begin{align}Y_{it} &= \mu+\tau_i+\varepsilon_{it}^0\\ &= (\mu+ \bar \tau_{\bullet})+(\tau- \bar \tau_{\bullet})+\varepsilon_{it}\\ &= \mu^\prime + \alpha_i + \varepsilon_{it}\tag 1.\end{align}

It can be shown that

\begin{align}\mathbb E[\textrm{SST}] &= \sum r_i\alpha_i^2 + (v-1)\sigma^2,\tag 2\label 2\\ \mathbb E[\textrm{SSE}]&= (n-v)\sigma^2\tag 3\label 3.\end{align}

Under the null hypothesis, $$\forall i, ~\alpha_i = 0;$$ so $$\eqref 2$$ becomes

$$\mathbb E[\textrm{SST}] = (v-1)\sigma^2;\tag 4\label 4$$

adding $$\eqref 4$$ and $$\eqref 3$$ yields

\begin{align}\mathbb E[\textrm{SST}+ \textrm{SSE}] &= \mathbb E[\textrm{SSE}_0]\\ &= (n-1)\sigma^2.\tag 5\end{align}

• This is not the reduced model. No doubt the calculations are similar, but I would want to know where, if anywhere, they differ. Also, the "It can be shown that" would need to be shown. Thanks for answering! Commented Dec 6, 2022 at 2:29
• Sure. I wanted to verify the result. Because you already correctly derived it. I merely showed that it was correct. And yes, if you want, I can show $(2), (3).$ Commented Dec 6, 2022 at 2:35