3
$\begingroup$

I'm working with a data set of 100 itens, 5 alternatives, medicine test and analyzing them by means of IRT (. The IRT - 3 parameter model yielded some values of b that are very large (|b| > 1000). I'm using IRTPRO application.

Searching the web I didn't find plausible answer to that.

The Baker(2001) - Item Response Theory - book says that theoretical values to parameter b is -4 < b < 4.

Someone with experience in IRT could explain me why I obtain such large values of b?

$\endgroup$
5
$\begingroup$

$\newcommand{\logit}{\rm logit}$The $b$ parameter, or so-called 'difficulty' parameter, is highly influenced by the slope of the estimated item (I'm not sure what IRTPRO calls them, but typically these are $\alpha$s or $a$s) and can take on values between $-\infty$ and $\infty$. If the slope values are close to 0 then the associated difficulty will become quite large, because $b$ is really talking about the inflection point along a monotonic probability response curve from the logistic (or ogive) family.

Most high-quality software do not estimate IRT models with the relationship $P = \logit(a(\theta - b))$ because it can be rather unstable when the discrimination parameters are too close to 0, and instead estimate the slope-intercept form $P = \logit(\alpha\theta + \delta)$ and transform the coefficients at a later stage. The transformations are of the form $a = \alpha$ and $b = -\delta / a$. Clearly when $a$ is very small a divide-by-zero issue begins to occur for the $b$ parameter, and indicates that the items' inflection point (location where the odds of answering correctly given some $\theta$ value is 50:50) is rather extreme.

It's very easy to see this in an example in R using a default dataset shipped with the mirt package:

> library(mirt)
> dat <- key2binary(SAT12,
+    key = c(1,4,5,2,3,1,2,1,3,1,2,4,2,1,5,3,4,4,1,4,3,3,4,1,3,5,1,3,1,5,4,5))
> mod <- mirt(dat, 1, itemtype = '2PL')
> coef(mod, simplify=TRUE)
$items
           a1      d g u
Item.1  0.801 -1.045 0 1
Item.2  1.502  0.438 0 1
Item.3  1.074 -1.141 0 1
Item.4  0.584 -0.530 0 1
Item.5  0.989  0.606 0 1
Item.6  1.148 -2.049 0 1
Item.7  1.004  1.383 0 1
Item.8  0.692 -1.508 0 1
Item.9  0.530  2.142 0 1
Item.10 1.007 -0.360 0 1
Item.11 1.732  5.249 0 1
Item.12 0.162 -0.345 0 1
Item.13 1.103  0.851 0 1
Item.14 1.036  1.174 0 1
Item.15 1.293  1.925 0 1
Item.16 0.726 -0.382 0 1
Item.17 1.550  4.164 0 1
Item.18 1.700 -0.851 0 1
Item.19 0.839  0.237 0 1
Item.20 1.537  2.610 0 1
Item.21 0.606  2.518 0 1
Item.22 1.540  3.479 0 1
Item.23 0.637 -0.850 0 1
Item.24 1.205  1.269 0 1
Item.25 0.771 -0.567 0 1
Item.26 1.534 -0.171 0 1
Item.27 1.916  2.770 0 1
Item.28 1.069  0.173 0 1
Item.29 0.835 -0.750 0 1
Item.30 0.386 -0.248 0 1
Item.31 2.336  2.784 0 1
Item.32 0.130 -1.652 0 1

$means
F1 
 0 

$cov
   F1
F1  1

Notice that item 32 has a very small slope in the default slope-intercept format. When changing to the classical IRT model, this results in a very extreme difficulty value.

> coef(mod, simplify=TRUE, IRTpars=TRUE)
$items
            a      b g u
Item.1  0.801  1.304 0 1
Item.2  1.502 -0.292 0 1
Item.3  1.074  1.063 0 1
Item.4  0.584  0.908 0 1
Item.5  0.989 -0.612 0 1
Item.6  1.148  1.785 0 1
Item.7  1.004 -1.378 0 1
Item.8  0.692  2.178 0 1
Item.9  0.530 -4.040 0 1
Item.10 1.007  0.358 0 1
Item.11 1.732 -3.030 0 1
Item.12 0.162  2.136 0 1
Item.13 1.103 -0.772 0 1
Item.14 1.036 -1.133 0 1
Item.15 1.293 -1.488 0 1
Item.16 0.726  0.526 0 1
Item.17 1.550 -2.687 0 1
Item.18 1.700  0.501 0 1
Item.19 0.839 -0.283 0 1
Item.20 1.537 -1.699 0 1
Item.21 0.606 -4.154 0 1
Item.22 1.540 -2.259 0 1
Item.23 0.637  1.334 0 1
Item.24 1.205 -1.053 0 1
Item.25 0.771  0.735 0 1
Item.26 1.534  0.112 0 1
Item.27 1.916 -1.446 0 1
Item.28 1.069 -0.162 0 1
Item.29 0.835  0.898 0 1
Item.30 0.386  0.643 0 1
Item.31 2.336 -1.192 0 1
Item.32 0.130 12.751 0 1

$means
F1 
 0 

$cov
   F1
F1  1

Finally, expanding the $\theta$ range in a plot to [-15, 15], you can see that the inflection point occurs around $\theta = 12.75$. Ultimately, this indicates that the item is extremely difficult, where even extremely bright individuals have very little chance of answering the item correctly. This is one of the main reasons why items with poor discrimination values are usually dropped from a test.

> itemplot(mod, 32, theta_lim = c(-15,15))

Item 32 probability plot

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.