3
$\begingroup$

I am analyzing data from a perceptual decision making experiment (10 participants, 1800 trials each). Participants made perceptual decisions (3 possible responses) and then rated their confidence on a continuous scale between 0 and 1.

My question is what predicts decision confidence, and whether the contribution of different variables interacts with response.

I am trying to fit a mixed effects model to my data (using Matlab's fitlme), with columns being:

  • s (subject index, categorical)
  • response (categorical, 3 options)
  • logRT (continuous)
  • correct (accuracy, binary)
  • evidence_sum (stimulus feature, continuous)
  • evidence_diff (stimulus feature, continuous)
  • confidence (continuous)

My effects of interest on confidence are the effects of logRT, correct, evidence_sum and evidence_diff on confidence, their interaction with response, and the main effect of response.

So I know the fixed effects part of my formula should be:

confidence ~ logRT + correct + response + evidence_sum + evidence_diff + logRT:response + correct:response + evidence_sum:response + evidence_diff:response

However, I am not sure about the random effects part. I assume I should have a random intercept term (1|s). Should I also include a random version (e.g., (response|sum)) for each of my fixed effects? If I don't, fixed effects become super-significant, but I guess that's just because I'm treating my data as if I have 18,000 trials with one participant, after mean-centering the confidence ratings of all participants? However, if I just include a random version for every effect of interest I worry that my effects of interest will be attributed the subject-specific regressors.

So my question is, given my research question of interest and given that I want to generalize my conclusions beyond my 10 participants, which random effects should I include in my model?

Thanks!

$\endgroup$
1
  • $\begingroup$ This looks like a multivariate problem to me. Correct and confidence (and maybe logRT) are also dependent on response and other variables, and are measured with error. You'll see that the problem is simplified if you treat it this way. $\endgroup$
    – vkehayas
    Jan 19, 2019 at 8:14

1 Answer 1

4
$\begingroup$

Since you have repeated measures on subjects and you are interested in generalising beyond the sample of subjects that you have, it makes sense to fit a model with random intercepts for subject.

When you also specify random slopes for a particular variable, you are allowing that variable to have a different influence on each subject, but there will still be an overall fixed effect for it. Whether or not this is a good idea depends on a few things: First, whether random slopes are supported by the theory of the underlying data generation process, second, whether they are supported by the data (that is, whether the random effects structure is estimable), and third, whether the inclusion of random slopes actually improves the fit, or not.

If a fixed effect is no longer significant after incorporating it into the random structure as a random slope, it means that the between-subject variability is sufficiently high that the overall slope is not significantly different from zero. In this case (indeed, even in other cases, but especially in this case) the variance of the random slopes may be of great interest. The distributional assumptions for the random effects (eg multivariate normality) should be checked, as a matter of course.

$\endgroup$
1
  • $\begingroup$ @TanZor if this answers your question, please can you mark it as an accepted answer. $\endgroup$ Jan 23, 2019 at 13:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.