Given the following experimental setup:

Multiple samples are taken from a subject and each sample is treated multiple ways (including a control treatment). What is mainly interesting is the difference between the control and each treatment.

I can think of two simple models for this data. With sample $i$, treatment $j$, treatment 0 being the control, let $Y_{ij}$ be the data, $\gamma_i$ be the baseline for sample $i$, $\delta_j$ be the difference for treatment $j$. The first model looks at both the control and difference:

$$ Y_{ij}=\gamma_i+\delta_j+\epsilon_{ij} $$ $$ \delta_0=0 $$

Whilst the second model only looks at the difference. If we precalculate $d_{ij}$ beforehand $$ d_{ij}=Y_{ij}-Y_{i0} $$ then $$ d_{ij}=\delta_j+\varepsilon_{ij} $$

My question is what are the fundamental differences between these two setups? In particular, if the levels are meaningless in themselves and only the difference matters, is the first model doing too much and is perhaps underpowered?

  • 2
    $\begingroup$ I can give a more thorough answer later, but I would suggest this paper by Paul Allison would be of interest (Allison, 1990). $\endgroup$ – Andy W Sep 2 '11 at 13:59
  • 1
    $\begingroup$ Edited to reflect the fact that the errors in the different models are not actually the same, and hence should not use the same symbols. $\endgroup$ – Rónán Daly Sep 2 '11 at 14:43

The $\epsilon_{ij}$ are likely to be correlated in the second model but not the first.

In the first, these terms represent measurement error and deviations from the additive model. With reasonable care--such as by randomizing the sequence of measurements--those errors can be made independent when the model is accurate. Whence

$$d_{ij} = Y_{ij} - Y_{i0} = \gamma_i + \delta_j + \epsilon_{ij} - (\gamma_i + \delta_0 + \epsilon_{i0}) = \delta_j + (\epsilon_{ij} - \epsilon_{i0}).$$

(Note that this contradicts the last equation in the question, because it is wrong to assume $\epsilon_{i0}=0$. Doing so would force us to concede that the $\gamma_i$ are random variables rather than parameters, at least once we acknowledge the possibility of measurement error for the control. This would lead to the same conclusions below.)

For $j, k \ne 0$, $j \ne k$ this implies

$$Cov(d_{ij}, d_{ik}) = Cov(\epsilon_{ij} - \epsilon_{i0}, \epsilon_{ik} - \epsilon_{i0}) = Var(\epsilon_{i0}) \ne 0.$$

The correlation can be substantial. For iid errors, a similar calculation shows it equals 0.5. Unless you are using procedures that explicitly and correctly handle this correlation, favor the first model over the second.

| cite | improve this answer | |
  • $\begingroup$ So, you've assumed that the first model is the true model and derived an undesirable property of the second model. We know all models are wrong so is this result really meaningful? $\endgroup$ – Macro Sep 2 '11 at 14:24
  • 1
    $\begingroup$ @Macro Please read my response more carefully: it is crafted to show what assumptions are needed to justify the first model and distinguish it from the second, but contains no assumptions that any model is "true." For instance, note the caveat "when the model is accurate." Even the word "accurate" was chosen with some thought towards avoiding the mis-impression that there is a "true" or "correct" model. $\endgroup$ – whuber Sep 2 '11 at 14:27
  • 1
    $\begingroup$ I'm a bit confused, what is $d_{ik}$? $\endgroup$ – Andy W Sep 2 '11 at 15:23
  • 1
    $\begingroup$ @Andy $j$ and $k$ index two distinct treatments. I should have written "For $j,k \ne 0$..."; I will fix that typo. Thanks for catching it. $\endgroup$ – whuber Sep 2 '11 at 15:47
  • $\begingroup$ @whuber Are there any references that support your statement, e.g. to convince reviewers? $\endgroup$ – Daniel Jan 28 '16 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.