0
$\begingroup$

I have a data set with an effort rating (varying in an ordinal scale from 1 to 7) as the response variable and a time point of measure (seven consecutive measures) as a fixed factor. An individual is a random intercept in my model. My data set has 48 participants. I have modeled the data using the ordinal-package´s clmm-function. This is my model:

effMod <- ordinal::clmm(effort_rating ~ measure_time + (1|Var1), data=NASA_effcomcat)

Is there a problem when my standard errors as well as the confidence intervals are super small? Here are the results:

model summary

confidence intervals

Here is what my data looks like. There is a rating of each individual on different time points.

data visualization

I really appreciate any help you can provide.

$\endgroup$

1 Answer 1

1
$\begingroup$

I recently had a similar experience where the standard error was far, far smaller than what I obtained without random effects, and the result of ordinal did not at all match a Bayesian random effects ordinal model. I concluded that the many approximations that are made in frequentist mixed effects models just don't work with this many random effects and this few subjects.

Bayesian random effects models are extremely easy to interpret and don't make approximations.

Random effects are not necessarily a well-fitting way to model within-person correlation. See https://fharrell.com/post/re

$\endgroup$
2
  • $\begingroup$ Thanks, Frank! Initially, the ordinal scale was from 1 to 21, but as there would have been too few observations in each rating, I combined each three consecutive levels to get the scale from 1 to 7. What I found a bit surprising was that when I combined some levels of the ordinal scale differently, the standard errors seemed way more normal. $\endgroup$
    – timothy
    Commented Sep 13, 2023 at 6:32
  • 1
    $\begingroup$ Since your model is assuming proportional odds (parallelism) it is not a good idea to combine levels and this should be totally unnecessary. If the number of levels causes a problem with ordinal that is a problem with ordinal. You can have thousands of levels with frequentist models using rms::orm (without random effects) or with Bayesian models using rmsb::blrm (with or without random effects). You don't combine levels with the Wilcoxon test and the PO model is a generalization of Wilcoxon. $\endgroup$ Commented Sep 13, 2023 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.