Questions tagged [irt]

The Item Response Theory framework encompasses a wide range of psychometric methods and mathematical models (latent trait models) to study individual responses to items defining a measurement instrument (multi-item scales such as questionnaires or tests) that is used to assess a specific construct (e.g., examinee proficiency or liability to depression).

80 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
800 views

Dealing with poor fit in an Item Response Theory model

I'm studying an online course with about 3000 students who each took several quizzes and I'm trying to apply Item Response Theory (using the ltm package in R) to model the questions, determine which ...
3
votes
0answers
59 views

IRT analysis on large sparse dataset

I have a dataset with 0/1 response data for a set of test questions. There are approx 7.5k items that have been randomly allocated to participants in an online quiz. Just over 10,000 participants (set ...
3
votes
0answers
269 views

Is Item Response Theory (IRT) a form of reinforcement learning?

I was doing studies in reinforcement learning and trying to apply it in Computerised Adaptive Testing (CAT). While learning IRT, I felt it is similar to behaviour of RL setting - receiving feedback ...
3
votes
0answers
52 views

When to use which ability estimates method?

There are several methods in IRT ability estimation,such as : expected a-posteriori ("EAP"), Bayes modal ("MAP"), weighted likelihood estimation ("WLE"), maximum likelihood ("ML"), expected a-...
2
votes
0answers
57 views
+50

How to incorporate multiple likelihoods in a probabilistic graphical model with Stan?

Data composition: In beta testing of a video game, users were assigned tasks in a many-to-many relationship. At the end of every day, users were asked to self evaluate (for each task) whether they ...
2
votes
0answers
36 views

Are there supposed to be bounds on parameters in 2PL Item Response Theory models?

Recently I've been studying Item Response Theory (IRT) and have come across some issues with the application side of it. I currently have a dataset of ~200 respondents x 7405 questions (quite ...
2
votes
0answers
29 views

How to best summarize Likert data (to use as an independent variable in a regression)?

I want to run a regression where one of the explanatory variables is a "summary" (details below) of a set of questionnaire questions that are answered on a Likert scale (although there are ...
2
votes
1answer
26 views

Can I do a repeated measures logistic regression for 16 students who answered different questions on a test?

I had 15 students take a vocabulary test with 100 items at the end of a course and were scored "1" (correct) or "0" (incorrect) for each item. I would like to predict the likelihood of them getting a ...
2
votes
0answers
45 views

Bayesian Model Fit for Binary Data

I am looking for methods (and preferably references) for assessing model fit for Bayesian analyses of binary data. Specifically, I am fitting Bayesian parametric and nonparametric item response models ...
2
votes
0answers
25 views

IRT vs proportion/frequency

Lets say I have response data from a multiple choice exam question and I want to understand whether the item was too difficult, or if I should change it somehow to improve it What benefits do I get ...
2
votes
0answers
32 views

Evaluation of experiment based on Multidimensional Item Response Theory?

For my master thesis within Educational Science and Technology, I have an experimental design with students divided over a control and an experimental group based on an educational intervention, and ...
2
votes
0answers
301 views

What does sample independence mean in Item Response Theory (IRT)

I keep reading that (at least the difficulty parameters of) an IRT model is sample independent. I find it difficult to get my head around this concept. Surely there must be limits to this statement. ...
2
votes
0answers
45 views

IRT modeling for questionnaire develop with a clinical:normal population of 15:1

I am trying to apply IRT to a questionnaire for screening purpose -- I have 3000 clinical population but only 200 normal population, would the result useful? The item plots and information function ...
2
votes
0answers
104 views

Likert scale survey student response independence

We administer a confidence survey (likert scale) comprising of 30 items and have conducted PCA to examine dimensionality, item independence and the like. We also conduct student interviews to assess ...
2
votes
0answers
55 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
2
votes
0answers
74 views

Are there local maxima in the standard 2-pl model or is there proof that there aren't?

I'm looking to use iterative logistic regression to find the parameter values of the 2 pl model: $ P = \sigma(\alpha_{i} (\theta_{s} - \beta_{i})) $ where $ \sigma(x) = \frac{1}{1+e^{-x}} $ I know ...
2
votes
0answers
171 views

Testing graded response model thresholds for significance?

I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ...
1
vote
0answers
50 views

Person's parameter (ability) calibration using R's package ltm

I have a user response ($0/1$) matrix user_item_matrix, which has $1000$ rows (users) and $30$ columns (items/questions). There is a lot of missing data (more than $...
1
vote
2answers
61 views

Using Co-variates in Item Response Theory 3PL model?

I am using Item Response theory(IRT) using 3 Parameter Logistic Model(3PL) for Logic test. After training the model, I use the posterior means of the item parameters 𝛼, β and γ to estimate person ...
1
vote
0answers
19 views

Downsides to “naive” estimation of longitudinal IRT factor scores?

I have a questionnaire administered at two time points. I would like to estimate a latent factor score (theta) at each administration using Item Response Theory. The ideal approach would be to model ...
1
vote
0answers
21 views

Bifactor model and infit statistics?

Good afternoon, I am currently in the process of calibrating an item bank using a GPCM model. So, am I right to assume that the bifactor model allows me to work with my general factor by ...
1
vote
0answers
70 views

How can I estimate student abilities with a Rasch Model?

I have a dataset of test results. There are 400 students, and 100 items. I ran ltm::rasch in R, and I computed ability scores for each student, and difficulty scores for each item. I would now like ...
1
vote
1answer
82 views

Poisson adaptation of IRT model: how to interpret parameters?

For my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get ...
1
vote
0answers
131 views

How to do unidimension test for graded response model?

Is there any method or R function can be used to check whether data is unidimensional with graded response model (grm)?
1
vote
0answers
509 views

Reporting LRT(log likelihood ratios) from anova()

I' ve got two logistic models/ based on IRT that I wanted to compare them using the log likelihoodratio test, so I used anova(model1, model2) in R. My output looks like following: ...
1
vote
0answers
208 views

Quantifying internal consistency with multiple raters and continuous rating scales

I have data from an assessment where people are rated by multiple raters on a set of criteria using a rating scale that is (essentially) interval. All people are rated by all raters. I believe that ...
1
vote
0answers
119 views

Longitudinal IRT estimation - wrong approaches?

I have been fitting longitudinal IRT models, checking for measurement invariance over time, constraining item parameters not to change over time, as recommended in the literature, and had good ...
1
vote
0answers
538 views

Simulating partial credit model polytomous IRT data using psych package for R

I see that the "psych" package for R includes many simulation options. I see in the help file that the user can specify the IRT model for the simulation, but I am not sure how to distinguish among the ...
1
vote
0answers
57 views

Where/when to implement replications in bootstrapping?

I have a dataset of people who completed a computer adaptive test (CAT), and calculated a person fit statistic for each examinee. Since the distribution of the statistic is unknown, I want to use a ...
1
vote
0answers
75 views

Consequences of violating assumptions about errors in IRT/Rasch based models

In the sorts of measurement models to which one might apply IRT models, including Rasch models, there is not thought to be an error associated with the latent variable. Rather, the latent variable is ...
1
vote
0answers
38 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave Kenny'...
1
vote
0answers
204 views

Recode Binary to scale Variables

I'm looking to find information to recode a set of dichotomous response variables to preferable a 1-5 scale. I have several response variables which when combined with PCA clearly depict certain ...
1
vote
0answers
650 views

Itemfit to IRT model with missing data

I have matrix of dichotomus correct/false answers with many random missing data. (The data comes from an ability test where questions were randomly drawn from an item bank.) I am trying to find out ...
1
vote
1answer
54 views

Reading recommendations for psychometric refinement

a research project turned up a large number of survey instruments on a specific topic. There is a great deal of conceptual item overlap in these instruments, but they do differ in some key ways, ...
1
vote
0answers
621 views

Longitudinal data format in WinBUGS or OpenBUGS, different number of items per time point

I am working with longitudinal data, where there are a different number of "items" per time point. I am working with the MCMCdynamicIRT1d function in ...
1
vote
0answers
236 views

Multigroup Longitudinal Item Response WinBUGS OpenBUGS

I was able to fit the Longitudinal IRT model in Winbugs for an ordinal response by extending the BUGS code I took from the paper by Curtis in JSS http://www.jstatsoft.org/v36/c01/paper/ However, I am ...
1
vote
0answers
700 views

Is it possible to fit a Graded Response Model in Stata?

I've been reading about Item Response Theory during the past few weeks and I'd like to use it to examine how my scales are functioning. The response categories are ordinal. If I understood well, the ...
1
vote
0answers
229 views

BUGS/JAGS for Nominal Response Model

I am a beginner with BUGS/JAGS and I was hoping to gather the opinion of you, experts. I am trying to implement a Nominal Item Response Analysis (Polytomous, not ordered, like a multiple-choice exam) ...
1
vote
0answers
39 views

Comparing the Goodness of fit of two polytomous set of items not nested

I'm working on two identical set of items that only differ in their frequency-of-occurence dimension (in the first is max.7 while in the second is max.4). They are both fit with the Generalised ...
1
vote
1answer
648 views

Overall goodness-of-fit/p-value for multiple items IRT model in R (ltm)

The data set which I am trying to analyse is Student Test Data. I have a data of responses (either 1/correct response or 0/incorrect response) on some questions of a set of students. I have fitted a "...
1
vote
0answers
98 views

How do I extract difficulty and discrimination from a non-parametric IRT model

I've been using the KernSmoothIRT module in R to do non-parametric analysis of some items. I'm trying to find a way to quantify the difficulty and discrimination ...
0
votes
0answers
12 views

moderated nonlinear factor analysis - also handles different Likert scales?

I have three datasets - one in which participants responded to the scale on a 5-point scale, two in which they did on a 7-point scale. Can I apply the MNLFA on these data? If I understood it correctly,...
0
votes
0answers
10 views

interpretation of item tree analysis plot

Two quick questions about Item-Tree analysis plot. (1) For the red plot, I know items {1L, 3L, 5L, 6L} are equally informative and thus not plotted. Does this mean ...
0
votes
1answer
60 views

IRT: What is enough information? And can a test provide too much?

I was wondering what makes for a good test information value in IRT, and came across this: Reliability: Item and test information functions graphically reflect how reliably the individual items and ...
0
votes
0answers
13 views

What is the source of the IRT 1PL discrimination constant?

For a 1PL IRT model you would use the 1.702 constant for the discrimination parameter. My questions are why, what is the source of this constant?
0
votes
0answers
43 views

Testing the fit of polychoric correlation

I’m using polychoric correlations for my work. According to Kampen and Weeren (2017): In order to prevent this questionable research practice, it is recommended that in applications of the ...
0
votes
0answers
18 views

Post-Processing Explanatory IRT Models in lme4

I have fit several different explanatory IRT models using lme4, and the final model includes item, person, and item-by-person covariates. The issue I'm now encountering is that I don't know how to get ...
0
votes
0answers
63 views

Difference between CFA using WLSMV estimator and IRT

I am trying to understand the differences (or similarities) between conducting CFA on categorical data using the WLSMV estimator in SEM and item response theory analysis. I would appreciate any input. ...
0
votes
0answers
15 views

Determine characteristics of test raters who systematically score higher or lower on a series of items

I have a survey that asks for respondents' subjective view on the presence of a marker (1/0). I have approximately 50 respondents and 40 marker questions. I want to tell what respondent ...
0
votes
1answer
55 views

Item response theory (IRT) for continuous responses

I hope this message finds everyone safe and well. I want to estimate the Rasch item difficulty parameters for my test items (dataset below). However, I have two challenges: (1) Item scores are ...