It seems to be a relatively common approach in some fields to, for a linear relationship which is subject to individual differences, estimate regression parameters separately for each subject in an experiment, and then use one sample t tests to check if each parameter is significantly different from $0$ across subjects.

Having done some work with mixed-effects models, however, it's not clear to me how this approach is a justifiable alternative to a mixed-model.

For example, Buchel et al (2011) report the following analysis

[...] we tested whether the risk taken (that is, the number of boxes opened) in round t + 1 can be predicted by the missed opportunity and the position of the finally opened box at round t (risk) [...]:

$$risk_{t+1}=c_0 + c_1missed\_opportunity_t + c_2 risk_t$$

[...] The estimated regression coefficients $c$ were then tested for significance in the whole group using a one sample t-test.

How does this approach compare to fitting an appropriate mixed-effects model?

lmer(risk ~ prev_missed_opportunity + prev_risk + (prev_missed_opportunity + prev_risk|subject))

$$risk_{t+1}=\\ \beta_0 + \gamma_{0,S} + \\ \beta_1 missed\_opportunity_t + \gamma_{1,S} missed\_opportunity_t +\\ \beta_2 risk_t + \gamma_{2,S} risk_t $$

where $\beta_{0..2}$ are the "fixed effects", and $\gamma_{0..2,S}$ are the normally distributed deviation from the fixed effect estimate for subject $S$.

Some points:

  • I understand that the mixed-effects model allows for partial pooling: coefficient estimates for participants who deviate from the rest of the group, or which have been estimated from a smaller sample, will shrink towards the population mean
  • Each participant's regression coefficient in the original paper is an estimate, rather than a known data point. Intuition says that this matters in some way, but I don't know how.
  • For what it's worth, in my own analysis, I've found that both methods result in roughly the same coefficient estimates, but the mixed-effects model yields lower standard errors, and thus lower p values.
  • A number of other SE questions have touched on this issue

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.