lme4 subject-level random effect for repeated measures but subjects belong to two different groups reflecting fixed effect

Question

I am analyzing effects for two different populations ('group membership') where each subject provided repeated measures. I would like to have the subject as a random effect given repeated measures, but subject-level intercept and slope may be influenced by the fixed-effect group membership, and so I would like the random effects to be evaluated within group versus across all subjects. What is the best way to approach this?

currently, for example, I have the below model:

glmer(choice ~ group*task+(task|subject),
     family = binomial,
     data = dat,
     glmerControl(optimizer="bobyqa"))

However, I've noticed that subject-level random effects are averaged across all participants and not within their particular group (when subject-level random effects are estimated within group, the values are quite different).

Thank you so much for your help!