# Internal consistency reliability in item response theory models

How can internal consistency reliability of a test and of individual test items be quantified in Item Response Theory models? I know I can resort to Classical Test Theory, Cronbach's alpha, and other measures, but is there a way to characterize reliability within IRT?

-

You can compute test information curves from your IRT parameter estimates. These curves give you the precision of the test at each $\theta$ of the latent trait. The information $I$ can be transformed into the standard error of estimate $SEE$, which is a direct estimate of the reliability of the test at that $\theta$: $SEE = 1 / \sqrt{I}$.

The metric of the test information can also be converted to a traditional reliability metric expressed by a correlation coefficient (Thissen, 2000): $Rel = 1 - (1/I)$. Here are the conversions from a set of TICs to correlational reliability estimates:

# following Thissen, 2000:
TIC <- seq(1, 12, by=1)
round((rel <- data.frame(TIC, SEE=sqrt(1/TIC), REL=1-1/TIC)), 2)

TIC  SEE  REL
1 1.00 0.00
2 0.71 0.50
3 0.58 0.67
4 0.50 0.75
5 0.45 0.80
6 0.41 0.83
7 0.38 0.86
8 0.35 0.88
9 0.33 0.89
10 0.32 0.90
11 0.30 0.91
12 0.29 0.92


For example, a TIC > 5 corresponds to a reliability > .80.

Thissen, D. (2000). Reliability and measurement precision. In H. Wainer (Ed.), Computerized adaptive testing: A primer (2nd ed., pp. 159–184). Lawrence Erlbaum Associates Publishers.

-
Thanks for your answer; I've asked a follow-up stats.stackexchange.com/questions/43405/item-information-in-irt – Jack Tanner Nov 12 '12 at 12:58