Data With Ordinal Responses: Calculate ICC & Assess Model Fit

Question

I have a data set of animal responses with repeated measures. The responses are on a 9-point scale. Here, I present an example data set that is very similar to mine:

library(dplyr)

set.seed(1234)

# number of subjects
n <- 300

# number of measurements per subject
k <- 4

# Assign a unique 4-digit ID number to each subject
rep(sample(1111:9999, n, replace = FALSE), each = k) %>%
  as.factor() -> id

# Generate random responses from subjects
c(replicate(n, c(sort(sample(1:9, 4, replace = TRUE))))) %>%
  as.numeric() -> score.numeric
  score.numeric %>%
  as.ordered() -> score.factor

# Assign sex to subjects
rep(gl(2, n/2, labels = c("male", "female")), each = k) %>%
  as.factor() -> sex

# Assign a generation to subjects
rep(gl(2, n/2, labels = c("p", "f1")), each = k) %>%
  as.factor() -> gen

# Mix the assignments within "gen"
gen <- sample(gen)

var.list <- list(id = id,
                 score.numeric = score.numeric,
                 score.factor = score.factor,
                 sex = sex,
                 gen = gen)

DF <- do.call(cbind.data.frame, var.list)

I employed the R package "ordinal":

model.clmm <- clmm(score.factor ~ sex + gen + (1 | id),
                   link = "cauchit",
                   Hess = TRUE,
                   data = DF)

I have two main questions:

1. Is it possible to obtain a repeatability value (ICC) with a 95% confidence interval using this package?

2. How do I analyze the fit of this model? I am new to ordinal models in general. My approach in the past with mixed models was to test the residuals for normality. It seems I cannot do the same with a clmm() model, or at least, doing so is not immediately obvious to me.

Answer 1

If you mean intraclass correlation coefficient when you mention ICC, as far as I know you can only calculate it by yourself as of today. According to Liu,2005, p.414, ICC= τ/(τ+3.29), where τ is between-group variance.

As of the assessing model fit part, you can provide -2LL, LR, delta(df), AIC, BIC, SABIC, log-likelihood, SABIC values, and also MuMIn Multi-Model Inference R package provide AICc value and model comparision functionality for ordinal clmm objects.