12
$\begingroup$

Given the following experimental setup:

Multiple samples are taken from a subject and each sample is treated multiple ways (including a control treatment). What is mainly interesting is the difference between the control and each treatment.

I can think of two simple models for this data. With sample $i$, treatment $j$, treatment 0 being the control, let $Y_{ij}$ be the data, $\gamma_i$ be the baseline for sample $i$, $\delta_j$ be the difference for treatment $j$. The first model looks at both the control and difference:

$$ Y_{ij}=\gamma_i+\delta_j+\epsilon_{ij} $$ $$ \delta_0=0 $$

Whilst the second model only looks at the difference. If we precalculate $d_{ij}$ beforehand $$ d_{ij}=Y_{ij}-Y_{i0} $$ then $$ d_{ij}=\delta_j+\varepsilon_{ij} $$

My question is what are the fundamental differences between these two setups? In particular, if the levels are meaningless in themselves and only the difference matters, is the first model doing too much and is perhaps underpowered?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I can give a more thorough answer later, but I would suggest this paper by Paul Allison would be of interest (Allison, 1990). $\endgroup$
    – Andy W
    Sep 2, 2011 at 13:59
  • 1
    $\begingroup$ Edited to reflect the fact that the errors in the different models are not actually the same, and hence should not use the same symbols. $\endgroup$
    – Rónán Daly
    Sep 2, 2011 at 14:43

1 Answer 1

Reset to default
7
$\begingroup$

The $\epsilon_{ij}$ are likely to be correlated in the second model but not the first.

In the first, these terms represent measurement error and deviations from the additive model. With reasonable care--such as by randomizing the sequence of measurements--those errors can be made independent when the model is accurate. Whence

$$d_{ij} = Y_{ij} - Y_{i0} = \gamma_i + \delta_j + \epsilon_{ij} - (\gamma_i + \delta_0 + \epsilon_{i0}) = \delta_j + (\epsilon_{ij} - \epsilon_{i0}).$$

(Note that this contradicts the last equation in the question, because it is wrong to assume $\epsilon_{i0}=0$. Doing so would force us to concede that the $\gamma_i$ are random variables rather than parameters, at least once we acknowledge the possibility of measurement error for the control. This would lead to the same conclusions below.)

For $j, k \ne 0$, $j \ne k$ this implies

$$Cov(d_{ij}, d_{ik}) = Cov(\epsilon_{ij} - \epsilon_{i0}, \epsilon_{ik} - \epsilon_{i0}) = Var(\epsilon_{i0}) \ne 0.$$

The correlation can be substantial. For iid errors, a similar calculation shows it equals 0.5. Unless you are using procedures that explicitly and correctly handle this correlation, favor the first model over the second.

Improve this answer
$\endgroup$
7
  • $\begingroup$ So, you've assumed that the first model is the true model and derived an undesirable property of the second model. We know all models are wrong so is this result really meaningful? $\endgroup$
    – Macro
    Sep 2, 2011 at 14:24
  • 1
    $\begingroup$ @Macro Please read my response more carefully: it is crafted to show what assumptions are needed to justify the first model and distinguish it from the second, but contains no assumptions that any model is "true." For instance, note the caveat "when the model is accurate." Even the word "accurate" was chosen with some thought towards avoiding the mis-impression that there is a "true" or "correct" model. $\endgroup$
    – whuber ♦
    Sep 2, 2011 at 14:27
  • 1
    $\begingroup$ I'm a bit confused, what is $d_{ik}$? $\endgroup$
    – Andy W
    Sep 2, 2011 at 15:23
  • 1
    $\begingroup$ @Andy $j$ and $k$ index two distinct treatments. I should have written "For $j,k \ne 0$..."; I will fix that typo. Thanks for catching it. $\endgroup$
    – whuber ♦
    Sep 2, 2011 at 15:47
  • $\begingroup$ @whuber Are there any references that support your statement, e.g. to convince reviewers? $\endgroup$
    – Daniel
    Jan 28, 2016 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.