3
$\begingroup$

I have a test where respondents are able to attempt each item as many times as required until they get it correct. I am looking to fit an IRT model to data from this test in order to look at the relative difficulties of different items, and how this maps onto person ability.

One possibility would be to think about the data in a binary sense of whether the person got the item correct in the first attempt. This would fit neatly onto the sort of 1PL/Rasch model that I'm well familiar with. However, it seems reasonable to assume that the number of attempts that the person takes in order to correctly respond is also related to the difficulty of the item (i.e., an item that generally takes 5 attempts to complete may be more difficult than one that generally takes 4).

Does anybody know of any extension to the Rasch model, or any other IRT model, that would allow me to estimate difficulty and ability parameters for a test of this sort?

$\endgroup$

1 Answer 1

2
$\begingroup$

Fitting a generalised partial credit/graded response model would work well here if you are sure of this phenomenon (if not, maybe consider a nominal response model for items you're unsure of). The idea is to represent the number of attempts it takes the person to answer as the 'observed item score', and use a monotonic IRT model to predict these category responses. So, if they answered correctly on the first try, score them as 1, or if it took 3 attempts, score as 3.

The only issue here is that the slopes associated with these items will be negative (because lower scores are associated with higher ability), so just flip them in your head to interpret the strength of the item. Otherwise, everything else in your IRT model should be the same.

$\endgroup$
2
  • $\begingroup$ Thanks for the idea - I had been so busy thinking about poisson and negative binomial distributions that I didn't think about it as a form of rating scale. I'm right in thinking that the signs of the slopes for person ability will be "wrong" too, right? $\endgroup$
    – Ian_Fin
    Jul 12, 2017 at 18:05
  • $\begingroup$ @Ian_Fin The person abilities should be on the correct scale, provided that the slopes for these items actually are negative (they could flip depending on the empirical identification....just check). So, they should be computed in the correct direction, and have the same interpretation as you are probably used to in other IRT models. $\endgroup$ Jul 12, 2017 at 18:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.