2
$\begingroup$

I hope this message finds everyone safe and well.

I want to estimate the Rasch item difficulty parameters for my test items (dataset below). However, I have two challenges:

(1) Item scores are continuous between 0 and 1 (item_score)

(2) Test is adaptive and thus, items (item_id) vary across persons (person_id)

Is there an R package to take (1) and (2) into account (I highly appreciate a demonstration)?

--Many thanks

#===== Dataset in R
dat <- read.csv('https://raw.githubusercontent.com/izeh/n/master/g.csv', stringsAsFactors = F)
$\endgroup$
2

2 Answers 2

1
$\begingroup$

Edit: This answer is not correct. It's not clear to me what OP is looking for, but take a look at the discussion at https://chat.stackexchange.com/rooms/107527/discussion-between-jeremy-miles-and-user7148318 before answering.

There is no package that can do this (as far as I know).

The lavaan package for sem does not give standard errors for latent variable estimates (and I don't know you can predict a model with new data).

You might be able to do this with merTools::predictInterval(), however I'm having trouble understanding your data. What identifies the question?

You see to have very large number of questions (~ 16000?) - each of which is not answered by very many people, so your standard errors will be high. Have I understood that correctly?

My guess is that the will look like:

library(dplyr)
library(lme4)
library(merTools)
dat <- read.csv('https://raw.githubusercontent.com/izeh/n/master/g.csv', stringsAsFactors = F)

dat$item_id <- factor(item_id)

dat_no_1 <- dat[dat$person_id != 1, ]
dat_no_1 <- dat[dat$person_id > 1, ]

fit1 <- lme4::lmer(item_score ~ as.factor(item_id) +
                     (1 | person_id),
                   data = dat_no_1)
system.time(
  pred1 <-
    merTools::predictInterval(fit1,
                              newdata = dat_no_1,
                              n.sims = 999)
)

I can't test this though, as I run out of memory.

Note that this ignores the non-normality issue.

$\endgroup$
6
  • $\begingroup$ First of all, thank you! so, the test is computer adaptive. It assigns items to test takers based on their response (i.e., if correct makes the next one harder, if incorrect make the next one easier). The item_id identifies the specific question targeted by each test taker. Each test for each person consists of 5 different item_types. Speaking of lavaan, it seems this page talks about a transformation on the responses and than using lavaan, do you think it makes sense for this data? BTW, what are the item difficulty parameters in your code? $\endgroup$ Commented May 2, 2020 at 20:49
  • $\begingroup$ Item difficulty parameters are the intercepts. The code i wrote is the first step - you get the item parameters (these are stored in fit1) then you predict the CI of person 1. If the CI is sufficienlty small, you can stop. If not, give them another item - so you'd need to wrap this in some sort of loop. The system.time() part is the time it takes to score the individual, I don't know how long this would take - if it's too long, presumably it's not feasible. $\endgroup$ Commented May 2, 2020 at 21:01
  • $\begingroup$ Do you really have 16k items? $\endgroup$ Commented May 2, 2020 at 21:02
  • $\begingroup$ !! OK, that's a lot. You definitely can't use lavaan for this. Can you run the code I put in the answer? (I can't on my current machine.) $\endgroup$ Commented May 2, 2020 at 21:17
  • $\begingroup$ You can't test the cat on any person who has taken an item that no other person has taken. $\endgroup$ Commented May 2, 2020 at 21:18
1
$\begingroup$

As noticed in the other answer by Jeremy Miles, IRT models can be thought as a care of random-effect models. This is discussed in greater detail by De Boeck et al:

De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., Tuerlinckx, F., & Partchev, I. (2011). The Estimation of Item Response Models with the lmer Function from the lme4 Package in R. Journal of Statistical Software, 39(12), 1–28. https://doi.org/10.18637/jss.v039.i12

In such a case 1PL model can be estimated in lme4 using the command

lmer(ir ~ -1 + item + (1 | id), data = DataSet, family = "binomial")

Since the items are in the unit interval, you still can use the binomial link as with binary responses. Crossed effects are also supported by mixed-effects models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.