The mathematical / statistical linear model,
$$y_{ij} = \mu + \rho_i + \tau_j + \epsilon_{ij},$$ where $\mu$ is the population mean, $\rho_i$ is the main effect of subject, $\tau_j$ is the main effect of treatment, $\epsilon_{ij}$ is the independent error (link, p. 2), describes a single-factor repeated measures design.
It is said that for the data collected in such a design we have to operate with the residuals calculated accordingly to the formula A
:
$$\hat\epsilon_{ij} = Y_{ij} - \bar Y_{i.} - \bar Y_{.j} + \bar Y.$$
The formula B
, on wiki is $\hat\epsilon_i = Y_i - \bar Y$.
My question is:
what formula is used to correctly calculate residuals in the model describing a single-factor repeated measures design to check assumptions (multivariate normality, homoscedasticity, sphericity, linearity)?
Thank you.