0
$\begingroup$

As far as I understand, item information tells us how well an item contributes to score estimation precision and it ranges between 0 and 1.

However, in my analysis (see the image), item 15 and item 6 seem to have very high item information values. Does that mean my model overfits the data at hand (30 items, 66 responses) and should these items be taken from the test? Their a-parameter values are 4.93 and 8.41 respectively. I appreciate any opinion.

enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

As far as I understand, item information tells us how well an item contributes to score estimation precision and it ranges between 0 and 1.

Close. I believe you are confusing item information with reliability. Item information $I(\theta)$ is not bounded between 0 and 1, though reliability is. Further, information conveys measurement quality conditional on a specific $\theta$ level, though reliability does not. @philchalmers makes this clear here.

However, in my analysis (see the image), item 15 and item 6 seem to have very high item information values. Does that mean my model overfits the data at hand (30 items, 66 responses) and should these items be taken from the test? Their a-parameter values are 4.93 and 8.41 respectively. I appreciate any opinion.

I agree with your assessment that IRT slope (i.e., what you call the "a-parameter") estimates greater than 4 are large. The first thing I would do would be to inspect their respective standard errors, which are likely quite large (both because your sample size is quite small, and also because of the values themselves). I would not delete them, however, as my guess would be their respective population values are still considerably larger than that of your other items.

Finally, it should be emphasized that your sample size of 66 is very small by IRT standards. Also, many of your items appear to have very small slopes, so if you were to remove any items, I would look to remove items with flat-looking item information curves (e.g., item 7, item 26, item 29, & item 30).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.