Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I just came across this paper, which describes how to compute the repeatability (aka. reliability) of a measurement via mixed effects modelling. The R code would be:

#fit the model
fit = lmer(dv~(1|unit),data=my_data)

#obtain the variance estimates
vc = VarCorr(fit)
residual_var = attr(vc,'sc')^2
intercept_var = attr(vc$id,'stddev')[1]^2

#compute the unadjusted repeatability
R = intercept_var/(intercept_var+residual_var)

#compute n0, the repeatability adjustment
n = as.data.frame(table(my_data$unit))
k = nrow(n)
N = sum(n$Freq)
n0 = (N-(sum(n$Freq^2)/N))/(k-1)

#compute the adjusted repeatability
Rn = R/(R+(1-R)/n0)

I believe that this approach can also be used to compute the reliability of effects (i.e. sum contrast effect of a variable with 2 levels), as in:

#make sure the effect variable has sum contrasts
contrasts(my_data$iv) = contr.sum

#fit the model
fit = lmer(dv~(iv|unit)+iv,data=my_data)

#obtain the variance estimates
vc = VarCorr(fit)
residual_var = attr(vc,'sc')^2
effect_var = attr(vc$id,'stddev')[2]^2

#compute the unadjusted repeatability
R = effect_var/(effect_var+residual_var)

#compute n0, the repeatability adjustment
n = as.data.frame(table(my_data$unit,my_data$iv))
k = nrow(n)
N = sum(n$Freq)
n0 = (N-(sum(n$Freq^2)/N))/(k-1)

#compute the adjusted repeatability
Rn = R/(R+(1-R)/n0)

Three questions:

  1. Do the above computations for obtaining the point estimate of the repeatability of an effect make sense?
  2. When I have multiple variables whose repeatability I want to estimate, adding them all to the same fit (e.g. lmer(dv~(iv1+iv2|unit)+iv1+iv2) seems to yield higher repeatability estimates than creating a separate model for each effect. This makes sense computationally to me, as inclusion of multiple effects will tend to decrease the residual variance, but I'm not positive that the resulting repeatability estimates are valid. Are they?
  3. The above cited paper suggests that likelihood profiling might help me obtain confidence intervals for the repeatability estimates, but so far as I can tell, confint(profile(fit)) only provides intervals for the intercept and effect variances, whereas I would additionally need the interval for the residual variance to compute the interval for the repeatability, no?
share|improve this question

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.