3
$\begingroup$

Some adaptive test systems (e.g. school assessment tools) use the 1pl IRT model, while others use the 2pl or the 3pl. When developing an adaptive IQ test, is there a rule of thumb about which model to choose in calibrating the item difficulty and test takers ability?

I can't find any research that gives some insights in fit between IQ test items and different kinds of IRT models.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

I think the difference primarily is a philosophical one when choosing Rasch/1PL models (the emphases on what measurement means is slightly different in that literature, and hence researchers try their best to obtain these special items), and an empirical/design one when deciding between using 2PL and 3PL models.

Since the slopes are all equal in 1PL models determining a persons location amounts to finding the optimal location where respondents have a P = 0.5 chance of answering correctly by simply choosing items with the best intercepts to get an estimate of $\theta$, whereas in 2- and 3PL models it's slightly more complicated due to the unequal slopes and lower bound parameters for guessing. As a consequence, 2-3PL models often require more advanced adaptive item selection procedures such as the Kullback–Leibler/Fisher information to select the next best item for honing in on $\theta$.

Speaking purely from a design perspective if the adaptive testing items contain a finite number of responses then the 3PL seems like the better option, but if it's more of a fill in the blank style answer (e.g., 2 + 3 = __.) then the 1PL and 2PL models would, at least theoretically, be more reasonable.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Phil, thank you for your answer! You mention researchers who ' try their best to obtain these special items'. Any literature I should read? One additional question: given the complexity of 2pl and 3pl, would you choose 1pl when wanting to make an adaptive test the easiest way? $\endgroup$
    – rdatasculptor
    Commented Jun 11, 2013 at 21:57
  • $\begingroup$ Oh and one (hopefully the last) additional question: would the fact that I am dealing with iq items instead of 'ordinary' math or knowledge items change which model to prefer? $\endgroup$
    – rdatasculptor
    Commented Jun 11, 2013 at 22:05
  • $\begingroup$ Measuring IQ isn't the most important characteristic per say, just that the items selected have the same correlation with the latent trait and hence can reasonably be fit by a Rasch-type model. However, I tend to think of the Rasch/1PL model as a kind of mythical unicorn that is hard to obtain in practice without a lot of discipline and patience. Formally, it's a nonlinear equivalent to having loading exactly equal in standard factor analysis (tau equiv). 2PL likely is what you should strive for, and if you can reduce to 1PL then do it via item diagnostics and nested model comparisons tests $\endgroup$
    – philchalmers
    Commented Jun 11, 2013 at 22:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.