I hope this message finds everyone safe and well.

I want to estimate the Rasch item difficulty parameters for my test items (dataset below). However, I have two challenges:

(1) Item scores are continuous between 0 and 1 (item_score)

(2) Test is adaptive and thus, items (item_id) vary across persons (person_id)

Is there an R package to take (1) and (2) into account (I highly appreciate a demonstration)?

--Many thanks

#===== Dataset in R
dat <- read.csv('https://raw.githubusercontent.com/izeh/n/master/g.csv', stringsAsFactors = F)

Edit: This answer is not correct. It's not clear to me what OP is looking for, but take a look at the discussion at https://chat.stackexchange.com/rooms/107527/discussion-between-jeremy-miles-and-user7148318 before answering.

There is no package that can do this (as far as I know).

The lavaan package for sem does not give standard errors for latent variable estimates (and I don't know you can predict a model with new data).

You might be able to do this with merTools::predictInterval(), however I'm having trouble understanding your data. What identifies the question?

You see to have very large number of questions (~ 16000?) - each of which is not answered by very many people, so your standard errors will be high. Have I understood that correctly?

My guess is that the will look like:

dat <- read.csv('https://raw.githubusercontent.com/izeh/n/master/g.csv', stringsAsFactors = F)

dat$item_id <- factor(item_id)

dat_no_1 <- dat[dat$person_id != 1, ]
dat_no_1 <- dat[dat$person_id > 1, ]

fit1 <- lme4::lmer(item_score ~ as.factor(item_id) +
                     (1 | person_id),
                   data = dat_no_1)
  pred1 <-
                              newdata = dat_no_1,
                              n.sims = 999)

I can't test this though, as I run out of memory.

Note that this ignores the non-normality issue.

  • $\begingroup$ First of all, thank you! so, the test is computer adaptive. It assigns items to test takers based on their response (i.e., if correct makes the next one harder, if incorrect make the next one easier). The item_id identifies the specific question targeted by each test taker. Each test for each person consists of 5 different item_types. Speaking of lavaan, it seems this page talks about a transformation on the responses and than using lavaan, do you think it makes sense for this data? BTW, what are the item difficulty parameters in your code? $\endgroup$ May 2 '20 at 20:49
  • $\begingroup$ Item difficulty parameters are the intercepts. The code i wrote is the first step - you get the item parameters (these are stored in fit1) then you predict the CI of person 1. If the CI is sufficienlty small, you can stop. If not, give them another item - so you'd need to wrap this in some sort of loop. The system.time() part is the time it takes to score the individual, I don't know how long this would take - if it's too long, presumably it's not feasible. $\endgroup$ May 2 '20 at 21:01
  • $\begingroup$ Do you really have 16k items? $\endgroup$ May 2 '20 at 21:02
  • $\begingroup$ !! OK, that's a lot. You definitely can't use lavaan for this. Can you run the code I put in the answer? (I can't on my current machine.) $\endgroup$ May 2 '20 at 21:17
  • $\begingroup$ You can't test the cat on any person who has taken an item that no other person has taken. $\endgroup$ May 2 '20 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.