I'm a neuroscientist trying to move away from frequentist to bayesian statistics, please bear with me...
I'm after a hypothesis test on some of my data: e.g., let's say I have reaction times for two conditions from 10 subjects, 100 trials per subject. All participants do both conditions, i.e. it is repeated measures.
I want to get a test for the difference between the two conditions. Normally, I would average reaction times for each condition in each subject and then do a paired (1-sample) t-test across difference between the two condition means in each subject.
With a Bayesian approach I could also average across conditions and then use the BayesFactor R Package to get a BF or Kruschke's BEST R package to get a region of practical equivalence (ROPE)...
However, I find the preceding step of averaging across the trials in each participant unsatisfactory for a Bayesian analysis: For example, if I just wanted a posterior measure of the response times for each condition across the group I'd do a hierarchical analsyis (using JAGS) and include all the data (data -> individual priors -> group prior -> hyperprior on group -> fixed hyperprior on the hyperprior). This way I can get posterior estimates of the group response time for each condition. However, I don't know how to 'test' if these distributions are different. I believe that this is made more complicated by the fact that the response time between conditions is correlated at the level of subjects (i.e. some are just naturally faster regardless of condition than others).
My question: is there a way to calculate a Bayes Factor at the group level using a hierarhical model using all the (correlated) data? Or, am I just over complicating things and should just use the methods I already know which average across trials as mentioned above.
I hope that makes sense.
PS. I'm aware that some people are not a fan of Bayes Factors, and I'm certainly open to other tests, however, personally they are a useful statistic for quick digestion of a result in a paper. I also know that some people are sceptical of testing a null of no effect, but again, I find this the most intuitive way to think about it - open to more sophisticated priors.