Questions tagged [irt]

The Item Response Theory framework encompasses a wide range of psychometric methods and mathematical models (latent trait models) to study individual responses to items defining a measurement instrument (multi-item scales such as questionnaires or tests) that is used to assess a specific construct (e.g., examinee proficiency or liability to depression).

Filter by
Sorted by
Tagged with
2
votes
1answer
22 views

item response theory - criteria for selecting items

I'm trying to use IRT to select the best items for a one-factor scale that has 20 items. I was wondering what I need to be looking at if I wanted to select 5 or so items from these items (for a short ...
7
votes
1answer
179 views

Analyzing a partially crossed design

I have a data set from a test (see below). The scoring algorithm gives each item (item_id) a score (y) that is continuous from $...
0
votes
0answers
14 views

Determine characteristics of test raters who systematically score higher or lower on a series of items

I have a survey that asks for respondents' subjective view on the presence of a marker (1/0). I have approximately 50 respondents and 40 marker questions. I want to tell what respondent ...
0
votes
1answer
45 views

Item response theory (IRT) for continuous responses

I hope this message finds everyone safe and well. I want to estimate the Rasch item difficulty parameters for my test items (dataset below). However, I have two challenges: (1) Item scores are ...
0
votes
0answers
14 views

Adding Post-stratified Weights to IRT 3PL Model

I am using IRT, 3 Parameter Logistic Model (3PL) for parameter estimation of my question parameters. As my data is biased I calculated Post-stratified weights using sample and population data. I want ...
0
votes
0answers
14 views

DIF: validity of carrying out a t-test on the final sample?

I have run a DIF analysis of a questionnaire, to see how autistic men and women answer differently, with a total sample of 530 autistic men and 530 autistic women. I have a general hypothesis that the ...
0
votes
0answers
18 views

DIF: match sample randomly or by age?

I am carrying out a DIF analysis comparing the scores of men and women on a questionnaire. A preliminary ANOVA revealed that when comparing total scores on the test, there is a significant effect of ...
0
votes
0answers
18 views

Latent Bayesian IRT model with unbalanced panels

I'm using JAGS to estimate an IRT model in which individuals appear multiple times in the dataset but where panels are unbalanced (some individuals appear more often than others. To give you a sense ...
0
votes
0answers
19 views

Differential Item Functioning with Autistic and control samples

I'm carrying out a DIF analysis of a self-report questionnaire which measures 'systemising', or the drive to construct/understand complex systems such as technology/maths/buildings/etc, to check for ...
0
votes
0answers
16 views

Wright-Map with multiple values for one item

To give some context: I'm quite new to quantitative research in social sciences and learned about IRT, Rasch Reliability and Wright-Maps yesterday. Today a colleague sent me this image: and asked ...
0
votes
0answers
24 views

Estimating IRT item parameters when ability is known

I'm reading the Fundamentals of IRT book by Hambleton et al. In this book they demonstrate the invariance of item parameters in a 3PL model by: First building the model on the full sample, deriving ...
1
vote
0answers
32 views

Person's parameter (ability) calibration using R's package ltm

I have a user response ($0/1$) matrix user_item_matrix, which has $1000$ rows (users) and $30$ columns (items/questions). There is a lot of missing data (more than $...
0
votes
1answer
38 views

Using Co-variates in Item Response Theory 3PL model?

I am using Item Response theory(IRT) using 3 Parameter Logistic Model(3PL) for Logic test. After training the model, I use the posterior means of the item parameters 𝛼, β and γ to estimate person ...
2
votes
1answer
26 views

Can I do a repeated measures logistic regression for 16 students who answered different questions on a test?

I had 15 students take a vocabulary test with 100 items at the end of a course and were scored "1" (correct) or "0" (incorrect) for each item. I would like to predict the likelihood of them getting a ...
0
votes
0answers
41 views

Relationship between parameters of Factor Analysis with categorical data and IRT-parameters

I have a problem in understanding the relationship between parameters of Factor Analysis with categorical paramters and IRT-parameters. The relationship is clearly defined, for example in Kamata, ...
0
votes
0answers
28 views

IRT degrees of freedom of item fit statistics

I'm fitting Rasch models with the mirt package. Upon measuring item-wise $Q_1$ using mirt::itemfit(model, fit_stats="X2"), I ...
1
vote
0answers
16 views

Downsides to “naive” estimation of longitudinal IRT factor scores?

I have a questionnaire administered at two time points. I would like to estimate a latent factor score (theta) at each administration using Item Response Theory. The ideal approach would be to model ...
1
vote
1answer
209 views

Difference between IRT and EFA to find factors

I am learning about Item Response Theory in which items are used to assess ability. In principle, multiple latent abilities may exist and some items test the one, while other items test another. This ...
0
votes
0answers
26 views

Specify correct answer in Nominal Response Model (IRT)

Unlike in the dichotomous model, it seems that nominal models are oblivious to which choice is the correct answer at any given item. However, I'd like to make sure that, as $theta\to\infty$, the ...
2
votes
1answer
141 views

Standard errors of parameters in IRT

DeMars's Item Response Theory makes the following claims about parameter estimation in 3PL models: The standard error of $b$ (difficulty) will be smaller when there are more examinees with $\theta$ (...
4
votes
1answer
341 views

Difference between empirical and marginal reliability of an IRT model

I am using the mirt library in R to fit an instrument (binary responses) comprising two dimensions. In the mirt documentation are mentioned two types of reliability. I would like to ask what is the ...
0
votes
0answers
23 views

Generalizability study of subscaled rating instrument

I have a school rating ($1$ thru $4$) instrument that consists of $9$ subscales (e.g., classroom instruction, school management etc.). Under each subscale, I have $6$ items on which rating occurs. I ...
0
votes
0answers
41 views

Anchor selection in Differential Item functioning in R

I have read several papers about anchor selection methods, but none of them mentioned R codes. I am wondering how to use R to get the initial anchor items from a test set since the selection of anchor ...
0
votes
0answers
25 views

How to calculate new ability estimate after answered question in 1PL MLE computer adaptive tesing model?

As you can probably tell from the question, I'm new to computer adaptive testing. I'm currently building a little computer adaptive test program using a 1P IRT curve. I've been reading a lot of ...
0
votes
0answers
19 views

Is Laud & Ibrahim's (1995) D(m) statistic a better model-fit indicator than the log-likelihood?

I am comparing model fit between a parametric (P) item response model and a Bayesian nonparametric (BNP) item response model. The log-likelihoods for the BNP model are more extreme than those for the ...
1
vote
1answer
59 views

Should subscales be analyzed separately under the IRT Graded Response model?

Suppose I have a Likert scale type questionnaire that assesses the amount of social support a patient receives. The questionnaire is divided into four subscales that measure: Emotional support ...
1
vote
1answer
76 views

Unidimensionality in IRT

I recently came across the definition of unidimensionality (in the context of IRT and internal consistency reliability, specifically Cronbach's alpha). Johnsosn (2007) states that There is a one-...
0
votes
1answer
126 views

Differential Item Functioning (DIF) Anchor Selection

I have a conceptual question regarding item anchor selection that seems to be so basic to the literature that authors of various online and print resources assume the reader understands. Namely, I am ...
1
vote
0answers
21 views

Bifactor model and infit statistics?

Good afternoon, I am currently in the process of calibrating an item bank using a GPCM model. So, am I right to assume that the bifactor model allows me to work with my general factor by ...
1
vote
1answer
197 views

How should we select between various Item Response Theory Models? (E.g. RSM, GRM, LRSM, PCM)

If you ask me what's the application scenario difference between NRM and GRM, I will answer ...
0
votes
1answer
80 views

Get different IRT model result when use functions from different R package?

Reproducible example as below: ...
0
votes
0answers
90 views

Testing for IRT assumptions with large samples and short scales

I’m using R’s mirt package to fit some (unidimensional) GPC models to a set of Likert-style items with five levels in each. I’m looking for some advice on how to test for the standard IRT assumptions. ...
2
votes
0answers
50 views

Why item discriminations are missing in IRT GRM, estimated as a part of the Structural Equation Model? [closed]

I have run a Structural Equation Model that has four latent constructs (FI, SC, EM and RB) in R. All of the factors are continuous. Meanwhile, indicators are all ordered categorical variables measured ...
1
vote
1answer
36 views

Item Response Theory model to recover the item locations

I can't seem to find a model to solve the following problem. It seems to be closely related to IRT, but not exactly. I have a list of users and items, I know the preference between users and items, ...
1
vote
1answer
204 views

Item response theory for continuous variables, and estimating standard error of measurement

I really like how traditional item response theory (IRT) packages tell you the standard error of measurement conditional on one's ability level, and from that, you can calculate the test information ...
3
votes
1answer
61 views

Combining multiple response models

Within a particular instrument, is it valid to combine multiple response models for subsets of items when estimating a latent trait? As an example, if I had some items that were conceptually suited ...
3
votes
0answers
49 views

IRT analysis on large sparse dataset

I have a dataset with 0/1 response data for a set of test questions. There are approx 7.5k items that have been randomly allocated to participants in an online quiz. Just over 10,000 participants (set ...
0
votes
1answer
86 views

Using IRT when most of the responses are null values

Greetings Cross Validated, I feel like I need to go into a little background before I bring up the specific issue I'm having. A few years ago Congress passed the FAST act, Part of this act mandated a ...
1
vote
0answers
67 views

How can I estimate student abilities with a Rasch Model?

I have a dataset of test results. There are 400 students, and 100 items. I ran ltm::rasch in R, and I computed ability scores for each student, and difficulty scores for each item. I would now like ...
0
votes
1answer
61 views

Couple questions regarding 3PL IRT model

It so happens we need to implement our own parameter estimation for 3PL IRT model in our small in-house tool we are developing. I'm using Stata which implements IRT to compare how our estimates fare ...
0
votes
1answer
401 views

How to convert IRT theta score to a percentage score

I am trying to implement an adaptive test using 3PL IRT model. We need to screen the candidates and label their expertise as ...
0
votes
1answer
97 views

3PL IRT model. Could someone point to a nice explanation and/or tutorial?

I'm very new to IRT. Even so, have just been asked to try and learn to use 3PL model on the data that we have. I'm completely out of my depth, and what's worse is we'll have to run our own parameter ...
0
votes
0answers
51 views

Why won't a 3PL IRT run?

I'm running some IRT models on test data and encountered a new problem. There are 36 items (scored as "correct" and "incorrect") and 287 participants with no missing data. I'm using Stata (not by ...
1
vote
1answer
78 views

Poisson adaptation of IRT model: how to interpret parameters?

For my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get ...
3
votes
2answers
161 views

In 1PL model, zero ability equals average ability, or change accuracy? [closed]

In a test evaluation scenario, latent variable analyses are used to represent the ability of the test-taker and the difficulty of the question. A 1PL model uses a latent, random log-odds threshold, $\...
0
votes
1answer
35 views

Ceiling effect of data-IRT Rasch EFA

Can you do IRT, CFA and/or Rasch if your data has items with large ceiling effects (i.e., no one was choosing the last two options of a 5-point likert scale)? Is it appropriate to do?
2
votes
1answer
42 views

How to proceed when collapsing ordinal items with levels that got no answer?

I have a set of 8 Likert-type items with 9 levels each. In no case do we have an item for which responses fill all possibilities and this has shown to be an issue for the SEM I'm trying to run (as ...
0
votes
0answers
48 views

Performance metric for math tasks

I have several math items which are varying in terms of difficulty. The difficulty ranges from zero to one, where zero is the most difficult one. Let's say users are solving such math tasks. I would ...
0
votes
1answer
172 views

IRT: Standard errors for intercepts vs. locations

In item response theory (IRT), the argument of the logit function can be expressed using a discrimination (a) and an intercept (b) as $a \theta + b$ or using a discrimination and location (c) as $a ( \...
0
votes
1answer
170 views

Convert ordinal to interval data with mirt package in R [closed]

I have ordinal data from 2 questionnaires. One questionnaire has a scale of 1:7 and the other of 1:5. I would like to convert these data to interval using polytomous item response theory. I can ...