2
$\begingroup$

In test theory, it appears to be widely recognized that very "easy" or "very difficult" items, relative to the ability of the sample of respondents, will have deflated correlations with the rest of the test (item-rest correlations). One way of explaining this, is that floor and ceiling effects will limit the variability of the responses, and hence any correlations with these. Item response theory would seem to take this into account. Looking at a sigma shaped item characteristic curve of e.g. a very easy item, data points are distributed at the upper right corner of the curve, where the slope is much less steep than at the middle of the curve. Item Characteristic Curve generated by RUMM2030 The above graph only illustrates this to some degree, but imagine that the dots were skewed even further to the right. It appears to me that the Rasch model (as well as other IRT models, I would assume) extrapolates the item response function across to the lower ability/person location levels, and estimates the item discrimination at the midpoint (the steepest part of the curve, where p=.5), where indeed no actual data points may exist. Would it be correct to say then, that the IRT item discrimination in this case offers a sample independent alternative to an item-rest correlation, an alternative that isn't affected by low variance? I've found plenty of references to the Rasch model being sample independent, but can't find anyone touching specifically on the above issue. The conclusion I read from the above ICC is that the model predicts the item to work well in a sample with lower trait level (θ). However, I would like to make sure this is an accurate conclusion, and preferably be able to back this statement up with a reference. Any thoughts?

$\endgroup$
  • $\begingroup$ This is pretty hard to read, at least on my phone. Consider some line breaks? The graph is much appreciated and this is a good question $\endgroup$ – shadowtalker Mar 26 '15 at 4:38
  • $\begingroup$ @ssdecontrol - Reads fine on my galaxy s4. $\endgroup$ – Erik Mar 26 '15 at 9:20
  • $\begingroup$ I think @ssdecontrol is referring to the flow of the text, in that it's hard to visually discern where and what your questions are, and what is important to read. Something more like this might be helpful to break up the text flow stats.stackexchange.com/questions/125287/… $\endgroup$ – philchalmers Mar 29 '15 at 16:06
4
$\begingroup$

To start, the Rasch model doesn't estimate any slope parameter in the model, and in fact treats them all as fixed (so called Tau-equivalent). Depending on how you parametrise the model will make this more apparent (i.e., fix all the discriminations to 1 and free the latent variance, or fix the latent variance to 1 and freely estimate the discriminations subject to the constraint that they are all equal).

Sample independence is largely overused in the Rasch literature and widely misinterpreted. You are correct in observing that as your points become more closely 'clustered' any use of item-test correlations will become problematic due to range restriction, but a similar thing happens in IRT as well (i.e., the variance in the first parametrisation method above will quickly approach 0). So you don't really solve the problem with IRT so much as introduce a different flavour of the phenomenon.

Regarding the interpolation to others outside of the observed range in your plot, it should behave well, but you are making an assumption that is outside the observable range of the test, and therefore in that particular range the item could be degenerate. This is exactly the same problem with extrapolating a linear regression line outside of the observed space....there's no guarantee or even evidence that it would work. Using the IRT parameters for a population that clearly was not sampled during calibration is known to have issues, so while, say, a high-school math test might work well for high-school students, giving the same test to lower ability participant (like in public school) is not guaranteed to work.

$\endgroup$
  • $\begingroup$ Thanks for a great answer. You are absolutely right, that the Rasch model doesn't estimate slope parameters. I think my original post was a bit misleading in this regard. However, most Rasch software should give you a clear indication of the relative slope, e.g. in RUMM2030 fit residuals will indicate this. Regarding the comment about the parametrisation approaching 0 (in the second paragraph) - Do you mean if all items suffer from range restriction? In my case, I mainly have range restriction on 2 or 3 out of 16 items. $\endgroup$ – Erik Apr 6 '15 at 20:47
  • $\begingroup$ @Erik what do you mean you have range restriction only on a few items, how would you know that? All you should know about those items is the item's slope and intercept. Your person parameters ($\theta$) should be distributed around N(0,1) if you parametrized the 2PL in the usual way, so I'm not sure the graph you are referring to above can happen. Can you elaborate or demonstrate what you are seeing? (might require an edit to your original question) $\endgroup$ – philchalmers Apr 6 '15 at 23:08
  • $\begingroup$ The person parameters of the scale will be centred at 0, as you say, but this is unlikely to be the case for the sample. In my case, the sample responds with high scores on the questionnaire and, accordingly, they are estimated to have a high degree of the latent trait. $\endgroup$ – Erik Sep 7 '16 at 7:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.