2
$\begingroup$

It seems to be a relatively common approach in some fields to, for a linear relationship which is subject to individual differences, estimate regression parameters separately for each subject in an experiment, and then use one sample t tests to check if each parameter is significantly different from $0$ across subjects.

Having done some work with mixed-effects models, however, it's not clear to me how this approach is a justifiable alternative to a mixed-model.

For example, Buchel et al (2011) report the following analysis

[...] we tested whether the risk taken (that is, the number of boxes opened) in round t + 1 can be predicted by the missed opportunity and the position of the finally opened box at round t (risk) [...]:

$$risk_{t+1}=c_0 + c_1missed\_opportunity_t + c_2 risk_t$$

[...] The estimated regression coefficients $c$ were then tested for significance in the whole group using a one sample t-test.

How does this approach compare to fitting an appropriate mixed-effects model?

lmer(risk ~ prev_missed_opportunity + prev_risk + (prev_missed_opportunity + prev_risk|subject))

$$risk_{t+1}=\\ \beta_0 + \gamma_{0,S} + \\ \beta_1 missed\_opportunity_t + \gamma_{1,S} missed\_opportunity_t +\\ \beta_2 risk_t + \gamma_{2,S} risk_t $$

where $\beta_{0..2}$ are the "fixed effects", and $\gamma_{0..2,S}$ are the normally distributed deviation from the fixed effect estimate for subject $S$.


Some points:

  • I understand that the mixed-effects model allows for partial pooling: coefficient estimates for participants who deviate from the rest of the group, or which have been estimated from a smaller sample, will shrink towards the population mean
  • Each participant's regression coefficient in the original paper is an estimate, rather than a known data point. Intuition says that this matters in some way, but I don't know how.
  • For what it's worth, in my own analysis, I've found that both methods result in roughly the same coefficient estimates, but the mixed-effects model yields lower standard errors, and thus lower p values.
  • A number of other SE questions have touched on this issue
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.