Let's say I'm running an experiment with an experimental condition and a control condition, where four subjects in total provide two reaction times each. I'm interested in estimating the treatment effect, and want to have varying intercepts for each subject.

Imagine the design is between-subjects, as depicted in the image below. enter image description here

If I'm not mistaken, I could model the $i$th reaction time of each subject ($S$) as $$ RT_{i,s,c}\sim \mathcal{N}(\mu_{s, c}, \sigma)\\ \mu_{s,c} = \alpha + \beta C + \gamma_s,\ \text{for}\ s\ \text{in}\ \{1, ..., 4\}\ \text{and}\ C\ \text{in}\ \{0, 1\} $$ with $$ \beta \sim \mathcal{N}(0 ,1)\\ \gamma_s \sim \mathcal{N}(0, 1) $$

So, I assume that each reaction time is drawn from a normal distribution whose mean is a function of an intercept/grand mean (a scalar $\alpha$), a deflection which represents the treatment effect (a scalar $\beta$), and a deflection which captures each subjects idiosyncracy (one of the four elements of the vector $\gamma_s$, depending on which subject's reaction time I'm modeling).

However, let's imagine I have a within-subjects design, where each subject provides two reaction times - one to the control condition and one to the experimental condition - as depicted below. enter image description here

How would I estimate the treatment effect ($\beta$ in the previous example)?

I can't move away from writing up a practically identical model: $$ RT_{i,c,s} \sim \mathcal{N}(\mu_{c,s}, \sigma)\\ \mu_{c,s} = \alpha + \beta C + \gamma_s\\ \beta \sim \mathcal{N}(0,1)\\ \gamma_s \sim \mathcal{N}(0,1) $$

Furthermore, I feel like the model could be conceived as in the image below: enter image description here

I'm interested in this because I've been thinking about how to model these kinds of experiments using a multilevel/hierarchical/mixed-effects model (or something akin to it). If I understand correctly, having the intercepts vary by subject is probably the most basic hierarchical model. However, I'm having trouble wrapping my head about what goes to which level of the hierarchy when dealing with within-subjects designs, and how to estimate the treatment effect.

I know that, for the between-subjects design, I could also drop the intercept and model $$ \mu_{s,c} = \beta_c + \gamma_s\\ \beta_c \sim \mathcal{N}(0,1)\\ \gamma_s \sim \mathcal{N}(0,1) $$ where $\beta_c$ would be a two-element vector, containing the mean of the distribution of control-condition data and the mean of the distribution of the experimental-condition data (instead of a deflection from the control condition as in the first parametrization).

If I'm not mistaken, I could also try to estimate the parameters of the distribution from which the subjects' deflections are drawn, instead of fixing it to $\mathcal{N}(0,1)$, i.e. $$ \gamma_s \sim \mathcal{N}(\mu_{{\gamma}_s}, \sigma_{{\gamma}_s})\\ \mu_{{\gamma}_s} \sim \mathcal{N}(0,1)\\ \sigma_{{\gamma}_s} \sim Exp(1) $$

This, however, leads me to believe that subjects should be at the top level of the hierarchy, and not the experimental condition. But, I kinda could do the same for the experimental conditions if I drop the intercept $$ \mu_{s,c} = \beta_c + \gamma_s\\ \beta_c \sim \mathcal{N}(\mu_{{\beta}_c},\sigma_{{\beta}_c})\\ \mu_{{\beta}_c} \sim \mathcal{N}(0,1)\\ \sigma_{{\beta}_c} \sim Exp(1) $$

I do, however, feel like this doesn't make a lot of sense, because maybe those two shouldn't be drawn from the same distribution.

In short, I'm not sure how to arrange the levels of the hierarchy for between- vs within-subjects designs, and whether I'm approaching effect estimation correctly (can it really be done with basically the same model?).

I think the details around the model specification can be ignored (e.g. vague priors, assuming normal distributions etc.).

I'm sorry if this is confusing, but I am, myself, confused. I'll appreciate any help, and will gladly answer all calls for clarification.

  • $\begingroup$ Is the first reaction time a "baseline" or "pre-condition" time? $\endgroup$
    – AdamO
    Feb 17, 2021 at 21:08
  • $\begingroup$ Let's assume the control represents the RT in absence of loud noises, and that the experimental RT is obtained under conditions of loud noise. So baseline, I guess? If I'm understanding the question correctly. $\endgroup$
    – Potato
    Feb 17, 2021 at 22:19
  • $\begingroup$ No. What's the point of two measurements within subject? $\endgroup$
    – AdamO
    Feb 17, 2021 at 22:23
  • $\begingroup$ Ah, sorry. It's common in cognitive psychology/psychophysics for each participant to provide K data-points when studying reaction times to stimuli. Here, K = 2 for simplicity. E.g. in the control condition, a subject could be shown a visual stimulus (e.g. square) in silence, and asked to press a button when the stimulus appear. In the exp. condition, the same could be done in presence of loud noises. Multiple reaction times per condition could be used to model the whole distribution of response times, capture variability, try to estimate the mean/median RT for each condition etc. $\endgroup$
    – Potato
    Feb 18, 2021 at 8:36


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.