Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Short question: How does the calculation and interpretation of IRT item information and test information change in the presence of item properties?

Long question: There's a variation on IRT called the linear logistic test model (LLTM):

$logit(Y_{p,j}) = \theta_p + \sum_{k=1}^K q_{j,k} \alpha_k$

For persons, $\theta_p$ is a random effect across persons $p$, just as in 1PL IRT. But unlike 1PL IRT, items have properties and each property is treated as a covariate. There are $K$ possible properties, and each item $j$ is coded with property values in the vector $q_{j,k}$. The effect of each property is the weight $\alpha_k$.

For example, if your test has math problems and reading problems, one item property may be an indicator of whether the item is a math problem or a reading problem.

Suppose the properties include indicator variables for each item $j$, i.e., that the LLTM item properties are a superset of the 1PL IRT item properties. That means we have a per-item effect, a.k.a. "difficulty" in IRT parlance. Knowing the difficulty of each item allows us to compute item information, and summing up information across item tells us the information of the full test.

In the presence of item properties, can we still talk about some questions being more informative than others? How does the calculation and interpretation of IRT item information and test information change in the presence of item properties?

share|improve this question
add comment

1 Answer

In terms of calculation, I don't really see a problem since for each item there is one random 'ability' component and really only one fixed item intercept (created from summing across the $K$ item predictor elements), so the calculation for each item is the standard Rasch information function $I_j(\theta, \beta_j) = P (1-P)$, where $\beta_j = \sum^K_{k=1} q_{j,k}\alpha_k$. The test information naturally is then just $T(\theta) = \sum^n_{j=1} I_j(\theta)$. This makes sense since the LLTM model is really just a design constrained dichotomous Rasch model.

The interpretation wouldn't be any different than a standard Rasch model either since by using the LLTM model you've selected a prior that the identified 'math' item block is systematically easier/more difficult than the 'reading' block, and constrained the model to reflect this by creating an appropriate item design matrix $q$. In turn this makes the model more parsimonious (saving precious degrees of freedom; though I'd be tempted to think of this as a bifactor model if it's the effect of the specific factors on $\theta$ you are worried about rather than a shift in item difficulty...).

Just a note, in this case no item is really more informative than others since they all have the same slope parameter by definition. However, they will be more informative at specific levels of $\theta$ (most informative at the point of inflection in the item response curve), which is no different than typical Rasch analysis interpretation.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.