Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How can internal consistency reliability of a test and of individual test items be quantified in Item Response Theory models? I know I can resort to Classical Test Theory, Cronbach's alpha, and other measures, but is there a way to characterize reliability within IRT?

share|improve this question
up vote 2 down vote accepted

You can compute test information curves from your IRT parameter estimates. These curves give you the precision of the test at each $\theta$ of the latent trait. The information $I$ can be transformed into the standard error of estimate $SEE$, which is a direct estimate of the reliability of the test at that $\theta$: $SEE = 1 / \sqrt{I}$.

The metric of the test information can also be converted to a traditional reliability metric expressed by a correlation coefficient (Thissen, 2000): $Rel = 1 - (1/I)$. Here are the conversions from a set of TICs to correlational reliability estimates:

# following Thissen, 2000:
TIC <- seq(1, 12, by=1)
round((rel <- data.frame(TIC, SEE=sqrt(1/TIC), REL=1-1/TIC)), 2)

  1 1.00 0.00
  2 0.71 0.50
  3 0.58 0.67
  4 0.50 0.75
  5 0.45 0.80
  6 0.41 0.83
  7 0.38 0.86
  8 0.35 0.88
  9 0.33 0.89
 10 0.32 0.90
 11 0.30 0.91
 12 0.29 0.92

For example, a TIC > 5 corresponds to a reliability > .80.

Thissen, D. (2000). Reliability and measurement precision. In H. Wainer (Ed.), Computerized adaptive testing: A primer (2nd ed., pp. 159–184). Lawrence Erlbaum Associates Publishers.

share|improve this answer
Thanks for your answer; I've asked a follow-up – Jack Tanner Nov 12 '12 at 12:58

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.