# Least Squares Estimation for CRD experiments

Setup

Let us use the following model for a Completely Randomised Designed (CRD) experiment:

$$Y_{ij}= \mu + \tau_i + \epsilon_{ij}$$ where the errors $$\epsilon _{ij}$$ ~ Normal ($$0, \sigma ^2$$) and are independent (and uncorrelated). Here, $$i$$ refers to a particular treatment where $$i = 1 . . .t$$ and $$j$$ refers to the $$j$$th observation of the $$i$$th treatment and $$j = 1 . . .n_i$$.

If we want to find Least Squares Estimates for $$\mu$$ or $$\tau _i$$ (for a fixed $$i$$) , then we need to minimise the sum of the squares of the errors. In other words, we want to minimise:

$$S =\sum _{i=1}^t \sum _{j=1}^{n_i} \epsilon _{ij}^2 = \sum _{i=1}^t \sum _{j=1}^{n_i} (Y_{ij}-\mu - \tau_i)^2$$

We can then look to find a suitable value of $$\mu$$ or $$\tau_i$$ (for a fixed $$i$$) respectively by trying to the value of that variable that minimises the sum.

Attempt

In order to minimise this sum, we first differentiate with respect to the variable of choice. If this is $$\mu$$, we get:

$$\frac{ \partial S}{\partial \mu} = -2 \sum _{i=1}^t \sum _{j=1}^{n_i} (Y_{ij} - \hat{\mu} - \hat{\tau}_i) = \color{red}{ -2 \sum _{i=1}^t \big{(}\sum _{j=1}^{n_i}(Y_{ij} - n_i \hat{\mu} - n_i \hat{\tau} _i) \big{)}} = 0$$

Question

My question is about what the justification is for the step in red.

I'm sure I'm overlooking something obvious, but I can't see why we can multiply $$\mu$$ and $$\tau _i$$ by these constants and have the sums remain equal. This is part of the solution in my textbook (and a similar principle is used for minimising with respect to $$\tau _i$$) and I would be interested in an explanation of why we are able to do this.

We have $$\frac{ \partial S}{\partial \mu} =0\implies -2 \sum _{i=1}^t \sum _{j=1}^{n_i} \left(Y_{ij} - \hat{\mu} - \hat{\tau}_i\right) = 0\tag 1\label 1$$

Expanding the sum in $$\eqref 1$$ yields the following normal equation:

\begin{align}\sum _{i=1}^t \sum _{j=1}^{n_i} Y_{ij}-\sum _{i=1}^t \sum _{j=1}^{n_i} \hat \mu-\sum _{i=1}^t \sum _{j=1}^{n_i}\hat \tau_i&=0\\\implies Y_{\cdot\cdot}-N\hat \mu -n_1\hat \tau_1-n_2\hat \tau_2 -\cdots -n_t\hat \tau_t&=0.\tag 2\label 2\end{align}

To see what's happening, we will go one summation at a time:

\begin{align}\sum _{i=1}^t \sum _{j=1}^{n_i} \left(Y_{ij} - \hat{\mu} - \hat{\tau}_i\right) &=\sum _{i=1}^t\left(\sum _{j=1}^{n_i} Y_{ij} - n_i\hat{\mu}-n_i\hat{\tau}_i\right) \\&= \sum _{i=1}^t \sum _{j=1}^{n_i} Y_{ij}-\sum _{i=1}^tn_i\hat{\mu}-\sum _{i=1}^tn_i\hat{\tau}_i\end{align} which leads to $$\eqref 2.$$

So, when they wrote $$\sum _{i=1}^t \left(\sum _{j=1}^{n_i}(Y_{ij} - n_i \hat{\mu} - n_i \hat{\tau} _i) \right) ,$$ it was an error/typo for $$\sum _{j=1}^{n_i}$$ hasn't been applied and yet there are $$n_i \hat{\mu} - n_i \hat{\tau} _i.$$

• I'm a little bit confused by your last line. I assume $Y_{\cdot \cdot}$ is defined as $\sum _{i=1}^t \sum_{j=1}^{n_i} Y_{ij}$ and that $N = n_1 + n_2 + . . . n_t$? Is that correct? Commented Feb 15, 2023 at 6:04
• Correct, it is. Commented Feb 15, 2023 at 6:08
• Similarly, solve for $\hat \tau_i$ to get the other normal equations. Commented Feb 15, 2023 at 6:08
• Your last line is in the form: $\big{(} \sum_{i=1}^t \sum_{j=1}^{n_i} Y_{ij} \big{)} - \sum_{i=1}^t (n_i \hat{ \mu}+ n_i \hat{\tau}_i) = 0$ but this isn't quite the same as the form of the line in red in the question - although maybe I'm missing something Commented Feb 15, 2023 at 6:19
• This is what is happening: you first operate the second summation, then the first summation. I am not seeing any intricacies here. And you are also stating the right things and yet you are not able to comprehend. So, why do you think the summation won't operate this way? Commented Feb 15, 2023 at 8:33