Questions tagged [item-response-theory]

Item Response Theory (IRT) is a set of statistical methods for modeling responses to test items. Use this tag for on-topic questions about (a) the theory itself (b) models appropriate for a specific case (c) implementing IRT in a specific tool such as an R package.

Filter by
Sorted by
Tagged with
26 votes
6 answers
20k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
Behacad's user avatar
  • 5,064
22 votes
3 answers
12k views

Item Response Theory vs Confirmatory Factor Analysis

I was wondering what the core, meaningful differences are between Item Response Theory and Confirmatory Factor Analysis. I understand that there are differences in the calculations (focusing more on ...
SimonsSchus's user avatar
11 votes
2 answers
3k views

Similarities and differences between IRT model and Logistic regression model

Despite the basic similarities like both of these model the probability of success rather than modelling the response variable directly; I believe that there are more reliable answers which point out ...
Artiga's user avatar
  • 333
9 votes
3 answers
4k views

What is Item Response Theory (IRT) called for continuous response?

I would like to model my problem using something similar to Item Response Theory, but my responses are not binary. They are continuous in $[0, 1]$. What are these models/the research field called?
user1141785's user avatar
9 votes
2 answers
10k views

Reliability in IRT Style

Reporting 'reliability' in IRT is more to 'informations' obtained and a bit different from a classical way using such as Cronbach alpha and KR20. Therefore, how to report the test reliability in ...
Hisham's user avatar
  • 111
8 votes
1 answer
974 views

Analyzing a partially crossed design

I have a data set from a test (see below). The scoring algorithm gives each item (item_id) a score (y) that is continuous from $...
Simon Harmel's user avatar
8 votes
4 answers
3k views

Simulating responses to a test for item-response theory

I am developing an online assessment system and I need to calibrate the bank of questions but I do not have enough people to implement a pilot test. That is why I decided to simulate the responses of ...
Erika Hernández's user avatar
8 votes
2 answers
7k views

Estimating ability using IRT when the model parameters are known

I have 3PL model parameters (guessing, difficulty and discrimination item parameters). Is there any function with which I can estimate individual ability from item response data? I tried the function <...
KH Kim's user avatar
  • 1,231
7 votes
2 answers
578 views

Discrimination/Slope Item Response Theory Models

In item response theory (IRT), 1-PL models, which include a slope parameter in addition to the intercept parameter, are better able to discriminate test-taker talent. Higher slopes translate to ...
jbuddy_13's user avatar
  • 3,010
7 votes
2 answers
1k views

What is the probability that a student gets a better score than another on a test with randomly selected questions?

Suppose there is a set $S$ of $100$ questions and there are $2$ students $a$ and $b$. Let $P_{ai}$ be the probability that $a$ answers the question $i$ correctly, and $P_{bi}$ the same for $b$. All $...
Daniel's user avatar
  • 205
7 votes
1 answer
621 views

Item reliability or fit in IRT-based adaptive tests?

My background is in machine learning and statistics, but I am relatively new to psychometrics and testing. Nearly all of the literature I've found on item reliability refers to Chronbach's alpha or ...
burr's user avatar
  • 358
6 votes
4 answers
3k views

Is R output reliable (specially IRT package ltm) [duplicate]

This may seem like a foolish question, but it is critical for me to have some expert advice on this. I am doing some research on students' learning levels in India. I am using R software (...
Deepak Agarwal's user avatar
6 votes
1 answer
326 views

Predict person parameters in explanatory IRT with lme4

I am using glmer from lm4 in R to run an explanatory IRT analysis. The aim is to model person parameters with items nested in persons and to control for age (the test scale has 10 items; all persons ...
Wolfgang Lenhard's user avatar
6 votes
1 answer
131 views

What is the role of non-anchor items when equating scales across studies using IRT?

I'm trying to use IRT to equate scales across 5 datasets. Three of them use the same scale (A); the other ones use different ones (B and C) There are items that overlap between A and C, A and B, but ...
llbia's user avatar
  • 331
6 votes
1 answer
3k views

Difference between empirical and marginal reliability of an IRT model

I am using the mirt library in R to fit an instrument (binary responses) comprising two dimensions. In the mirt documentation are mentioned two types of reliability. I would like to ask what is the ...
Nikolaos's user avatar
6 votes
1 answer
3k views

Internal consistency reliability in item response theory models

How can internal consistency reliability of a test and of individual test items be quantified in Item Response Theory models? I know I can resort to Classical Test Theory, Cronbach's alpha, and other ...
Jack Tanner's user avatar
  • 4,842
6 votes
2 answers
4k views

Longitudinal item response theory models in R

I'm trying to fit longitudinal item response theory (IRT) models in R. I have a test that was administered at multiple measurement occasions. I'd like to examine individuals' growth curves of factor ...
itpetersen's user avatar
6 votes
2 answers
3k views

IRT in R: Does anyone know of an IRT item calibration function that can cope with NA's?

I have a dataset that looks like this. ...
rdatasculptor's user avatar
6 votes
1 answer
340 views

BIC in Item Response Theory Models: Using log(N) vs log(N*I) as a weight

In IRT software packages and in the literature it is common to calculate the BIC as $$ \mathrm{BIC} = -2 \cdot \mathrm{logLik} + \log(N)\mathrm{Npars} $$ where $N$ is the number of rows in wide ...
Philipp's user avatar
  • 452
6 votes
1 answer
4k views

How can I handle missing values when fitting IRT model?

I am conducting a study for analyzing male involvement in Family planning. It is of interest to develop an index for the involvement of male. I am currently having difficulties fitting an IRT model ...
paualdemita's user avatar
6 votes
1 answer
350 views

IRT/Rasch modeling with very large N

I want to fit a 1-parameter IRT model on a questionaire with 15 questions and about six million people. Considering the large N, standard errors aren't essential. It looks like the IRT world is sort ...
DavidShor's user avatar
  • 1,491
5 votes
2 answers
2k views

theta parameter in R's LTM package

I am curious whether there is an intuitive way in ltm package in R to display person's estimated latent trait after fitting the ...
Carl H's user avatar
  • 512
5 votes
2 answers
102 views

Finding a Common Thread in Disparate Indicators

All of the theory I know about factor analysis, scales, etc revolves around separating out indicators into groups that each have (ideally) one underlying dimension. I have a student for whom we'd ...
justme's user avatar
  • 775
5 votes
4 answers
710 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
KH Kim's user avatar
  • 1,231
5 votes
2 answers
267 views

Dealing with least and maximum score in IRT

Generally when I do IRT modelling (using R software, 2PL model, ltm package) I remove records of students who have scored either 0 or 100%. The logic being that we do not have enough information about ...
Deepak Agarwal's user avatar
5 votes
1 answer
4k views

How to check goodness of fit for a graded response model in R

I am using the grm function in the ltm package in R. There is no function to check the goodness of fit of the output. How can I ...
Deepak Agarwal's user avatar
5 votes
1 answer
2k views

How to extract factor scores for each observation in IRT? [closed]

library(ltm) fit <- rasch(LSAT) factor.scores(fit) This code can generate latent trait scores for response patterns. How can I extract latent trait scores for ...
Rach's user avatar
  • 51
5 votes
1 answer
2k views

Difference between IRT and EFA to find factors

I am learning about Item Response Theory in which items are used to assess ability. In principle, multiple latent abilities may exist and some items test the one, while other items test another. This ...
LBogaardt's user avatar
  • 582
5 votes
1 answer
372 views

Is Item Response Theory (IRT) a form of reinforcement learning?

I was doing studies in reinforcement learning and trying to apply it in Computerised Adaptive Testing (CAT). While learning IRT, I felt it is similar to behaviour of RL setting - receiving feedback ...
Aljo Jose's user avatar
  • 523
4 votes
3 answers
3k views

Is there an R package which implements weighted maximum likelihood for Rasch models?

Is there a package in R which implements the weighted maximum likelihood method (Warm, 1996) for estimating the person parameters in Rasch Models?
Paul Illg's user avatar
4 votes
2 answers
2k views

Meaning of large values of parameter b (|b| >>) 4 in IRT theory

I'm working with a data set of 100 itens, 5 alternatives, medicine test and analyzing them by means of IRT (. The IRT - 3 parameter model yielded some values of b that are very large (...
Caco's user avatar
  • 143
4 votes
3 answers
443 views

Applications of Item Response Theory

As far as I know, item response theory is mainly used within psychology to analyse survey data, personality questionaires or in educational setting. Are there other areas (possibly outside of ...
Jens Kouros's user avatar
  • 1,075
4 votes
2 answers
795 views

Latent Class Analysis vs Rasch Analysis

I was wondering if anybody could explain the core, philosophical/conceptual differences between Latent Class Analysis and Rasch Analysis. I'm not necessarily referring to the exact mathematical ...
SimonsSchus's user avatar
4 votes
1 answer
378 views

Is there a way to place a priori constraints on item difficulty under item response theory?

I have a dataset where the difficulty level of my items and skill of my participants is unknown, but I would like to be able to extract some metric of how individual participants do on the test. What ...
russellpierce's user avatar
4 votes
1 answer
336 views

How many parameters are estimated in a 2PL model?

I'm familiar with 1PL IRT models, where you have some data matrix, and you model the latent factors, theta (the trait/ability) and (item) difficulty. ...
jbuddy_13's user avatar
  • 3,010
4 votes
2 answers
4k views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
Daniel's user avatar
  • 53
4 votes
1 answer
3k views

Interpreting discrimination values in IRT polytomous (grm) models. Are there any cut-offs or anchors?

I am working on a project that is looking at the discriminative properties of 6, 5-point Likert items in a scale. I am using the ltm package in R to examine this within a item response theory (irt) ...
Xander's user avatar
  • 416
4 votes
1 answer
362 views

how to select IRT model based on anova?

I have my2pl as ltm data and my3pl as tpm data. ...
kittygirl's user avatar
  • 294
4 votes
1 answer
1k views

Questions Regarding Item Response Models – the 3PL model vs. the mixture of 2PL and 3PL model

I currently work on an item analysis for an assessment. The assessment consists of 25 multiple-choice questions. Each question has 4 choices with one correct answer. I used different IRT models to ...
westjourney's user avatar
4 votes
1 answer
609 views

Item information in IRT

According to item information curves, item information for a 2PL IRT model is $I(\theta)=a^2_i p_i(\theta) q_i(\theta)$ To determine $p_i(\theta)$ and $q_i(\theta)$, do you just use the observed ...
Jack Tanner's user avatar
  • 4,842
4 votes
1 answer
324 views

Do 1PL IRT models measure both ability and difficulty, or just difficulty?

I'm trying to better understand Item response Theory (IRT) from a Bayesian perspective. Hypothetically, suppose I want to use a 1PL model and my data is a binary matrix ...
jbuddy_13's user avatar
  • 3,010
4 votes
1 answer
112 views

Methods for drawing population inferences from multiple sub-population datasets

What would be an appropriate model or method for making inferences about a broader population quantity from multiple quantities representing subsets of the population? Imagine, as an example, that I ...
Dr. Beeblebrox's user avatar
4 votes
0 answers
223 views

IRT analysis on large sparse dataset

I have a dataset with 0/1 response data for a set of test questions. There are approximately 7.5k items that have been randomly allocated to participants in an online quiz. Just over 10,000 ...
Jeanne's user avatar
  • 41
4 votes
0 answers
1k views

Dealing with poor fit in an Item Response Theory model

I'm studying an online course with about 3000 students who each took several quizzes and I'm trying to apply Item Response Theory (using the ltm package in R) to model the questions, determine which ...
Chiara Coetzee's user avatar
3 votes
2 answers
209 views

In 1PL model, zero ability equals average ability, or change accuracy? [closed]

In a test evaluation scenario, latent variable analyses are used to represent the ability of the test-taker and the difficulty of the question. A 1PL model uses a latent, random log-odds threshold, $\...
Carrot's user avatar
  • 33
3 votes
1 answer
649 views

Is it acceptable/common practice to report IRT theta scores as a percentile?

I have been teaching myself IRT. I think I get it, however, I've not yet reached a point where I would say that I'm totally confident. In assuming that the latent trait is normally distributed with ...
faustus's user avatar
  • 265
3 votes
1 answer
872 views

Is it possible to do IRT distractor analysis using R?

I've been using the mirt package to analyse a few response matrixes I have through IRT. However, every time I need to perform distractor analysis, I end up having ...
Douglas De Rizzo Meneghetti's user avatar
3 votes
1 answer
1k views

Item response theory vs. summative Likert scale

If the items to be "summed" or combined to create an overall index are collectively the underlying construct (i.e. I am trying to measure compliance to an intervention which has different components), ...
oncearunner's user avatar
3 votes
1 answer
2k views

How is a bifactor IRT model different from a factor analysis?

How is a bifactor IRT model different from a factor analysis? How would you describe their key differences? Any references you could direct me to would be very helpful. Thank you for sharing your ...
Xander's user avatar
  • 416
3 votes
1 answer
175 views

Pros/cons of estimating parameters for missing observations?

Some people are playing a game online. Every time a person plays, a new game board is generated randomly. On generation of a new board, the player can also choose a special weapon. (The choice of ...
Jack Tanner's user avatar
  • 4,842

1
2 3 4 5 6