I have a dataset with 0/1 response data for a set of test questions. There are approx 7.5k items that have been randomly allocated to participants in an online quiz. Just over 10,000 participants (set to increase to around 25,000) have responded to varying numbers of questions. The participants can choose how many questions to answer. The number of responses given generally ranges between 20-70 responses, although some up to around 200. The majority of cells in the data set are therefore blank.

My challenge is to get difficulty estimates so that I can put the items in an ordered list. Whilst I am confident handling this type of analysis for smaller datasets and have used packages in R such as ltm or psych, I am struggling to get any results with these data. I have considered dividing into subgroups and using anchor values, but the random allocation of items means there are no clear subsets with reasonable numbers of responses from individual participants.

Any suggestions on how I can work with this to derive comparable difficulty estimates for the items?

  • $\begingroup$ What is a difficulty estimate? How and what do you want to order? $\endgroup$ – user2974951 Feb 14 '19 at 13:03
  • $\begingroup$ Hi - the difficulty estimate is the item parameter in the IRT model. By the ordering I simply mean the most easy to most difficult questions based on these estimates. $\endgroup$ – Jeanne Feb 14 '19 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.