Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ratings (i.e. I believe I'm interested in whether the item is subject to differential item functioning, DIF).

There does not seem to be a straightforward way to ask if a single item is subject to DIF; rather, the existing methods I can find (e.g. the R package lordif) take a collection of multiple items (at least five items in the case of lordif) and responses and determine whether any of those items is subject to DIF.

I've fit graded response models to each of the two raters' answers individually to produce item characteristic curve plots and visually the thresholds for use of the five ratings look distinct from one another but I'd like a way to test these for significance.

Can anyone provide guidance on testing single items for DIF? Is this even a meaningful question in the absence of a second 'anchor' item for which no DIF is assumed?

share|improve this question
Yes, you need to use anchor items to have a common scale for latent trait. But isn't your question more about inter-rater variability? How many subjects and items do you have? – chl Nov 25 '12 at 21:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.