What is the variance decomposition method?

Question

For $i = 1, \ldots, m$ and $j = 1, \ldots , n$ we have observations $x_{ij}$. We can assume that $$ x_{ij} = y_{i} + z_{ij}, \qquad y_{i} \sim \mathcal{N}(\mu_{y},\sigma_{y}^{2}), \quad z_{ij} \sim \mathcal{N}(0,\sigma_{z}^{2}). $$ If necessary, we can also assume that $y_{i}$ and $z_{ij}$ are independent (but if this is not necessary for my question I would rather not make this assumption). The statistics text I am reading makes the claim that $\sigma_{y}^{2}$ and $\sigma_{z}^{2}$ can be estimated by the variance decomposition method (without reference).

I would appreciate any useful reference or explanation how this method works. I think it is related to ANOVA or factor analysis, but I could not find something which looks applicable in my particular setting, because I only have observations $x_{ij}$.

Answer 1

Not sure what your book is referring to, but it would seem to me that if you estimated variance at fixed $i$:

$$ Var\left[x_{ij}|i=const\right]=Var\left[z_{ij}|i=const\right]\approx\sigma_z^2,\,\mbox{at some }i $$

So you should be able to estimate the variance of $z$ by stratifying on $i$ (i.e. keeping $i$ fixed). You will get multiple variances in this way, it will then make sense to average them

$$ E\left[Var[x_{ij}\:|\:i]\right]\approx\sigma_z^2 $$

Here $E$ stands for the expectated value

You can then introduce:

$$ v_i=E\left[x_{ij},\: | \:i=const\right]=y_i+\zeta_i,\quad\zeta_i=E\left[z_{ij},\: | \:i=const\right]\sim\mathcal{N}\left(0,\,\frac{\sigma_z^2}{N_i}\right) $$

Where $N_i$ is the number of $j$-samples you have for a specific $i$. Then:

$$ Var[v_i]\approx \sigma^2_y+\frac{1}{M-1}\sum_i \frac{\sigma_z^2}{N_i} $$

If $x_i$ and $z_{ij}$ are independent. You can compute the LHS, and you know the variance of $z$ on RHS, so should be able to get the estimate of $y$. $M$ is the number of different i-values you have