What level to use when comparing subjects in a hierarchical Bayesian analysis?

Question

Say that I have an experiment where I test the reaction time of a number of subjects where each subject makes many reaction time trials. In a Bayesian framework the reaction times ($y$) could be modeled by a hierarchical model with prior distribution both on the subject level and for the whole group of subjects. A diagram of the model, Kruschke style, could be:

... and the corresponding BUGS/JAGS code would be:

for(i in 1:length(y)) {
  y[i] ~ dnorm(mu[subj[i]], tau[subj[i]])
}

for(j in 1:nbr_of_subjects)
  mu[subj[i]] ~ dnorm(M_mu, P_mu)
  tau[subj[i]] ~ dgamma(S_tau, R_tau)
}

M_mu ~ dnorm(M_M, P_M)
P_mu ~ dgamma(S_P, R_P)

S_tau <- pow(m , 2) / pow(sd, 2)
R_tau <- m / pow(sd, 2)
m ~ dgamma(S_m, R_m)
sd ~ dgamma(S_sd, R_sd)

If I wanted to compare the reaction time of two subjects I would then compare the their respective $\mu$ distributions. If the reaction time trials were split up into four blocks I could also model that by adding an extra block level with priors between the subject level and trial level in the diagram (as it might be the case that the subjects reaction time differs slightly between blocks for some reason).

My question is now, if I would want to compare two subjects what distributions should I compare? I could compare the distribution of the means on the subject level (which now partly defines the prior for the mean on the block level) but I could also compare the distribution of the means on the block level which corresponds to $\mu$ in the old model. In one way it seems more logical to compare the subjects on the subject level, but does it make any difference? And if there are very few blocks, say two, wouldn't the distribution of the means on the subject level be very "wide"?

Answer 1

Just to clarify your model

Let $y_{ij}$ be reaction time for participant $i$ on trial $j$.

$$y_{ij} \sim N(\mu_i, \sigma^2_i)$$

And then you model $\mu_i$ and $\sigma^2_i$ as coming from some other distribution with hyperparameters.

You ask if I would want to compare two subjects what distributions should I compare?

So for example, if you wanted to compare subject $i=1$ to subject $i=2$. Then you would have the estimated degree to which participant 2 had a greater mean as: $$\Delta\mu_{1,2} = \mu_2 - \mu_1$$

Alternatively, you could have the increase in estimated standard deviation as:

$$\Delta\sigma_{1,2} = \sigma_2 - \sigma_1$$

Naturally, under normal conditions the more observations you have per individual, the more precise your estimates of $\mu_i$ and $\sigma_i$ will be. And as a consequence your estimates of $\Delta\mu$ and $\Delta\sigma$ will also improve.

I don't quite understand what you are asking about blocks and trials.