Questions tagged [item-response-theory]

Item Response Theory (IRT) is a set of statistical methods for modeling responses to test items. Use this tag for on-topic questions about (a) the theory itself (b) models appropriate for a specific case (c) implementing IRT in a specific tool such as an R package.

Filter by
Sorted by
Tagged with
0 votes
0 answers
25 views

Rasch Analysis: comparison of the distribution person location estimates across independent samples

I hardly understand Rasch analysis (and this isn't my goal) but I need to clarify the following questions before delving deeper: Does it make any sense within the framework of Rasch analysis (and ...
user avatar
  • 101
1 vote
1 answer
27 views

IRT when each exam has a different set of questions from a common pool

I have a dataset with 0/1 response data for a set of test questions. There are approx 7.5k items that have been randomly allocated to participants in an online quiz. Just over 10,000 participants (set ...
user avatar
0 votes
0 answers
33 views

IRT - Aligning scores on a scale of 3PL to 1PL

I am facing some issues in a project regarding an alignment process. If you are willing to help me somehow, I thank you in advance. I will contextualize the problem below since it is easier to ...
user avatar
  • 1
1 vote
0 answers
12 views

Using IRT, how would you find the updated ability of someone after answering a calibrated item?

For this project, I have a list of items that were calibrated under the 3PL model using the mirt package. I had a pilot study run to help estimate these items. Now, using a different group of people, ...
user avatar
1 vote
0 answers
18 views

Which item to administer to improve the standard error of the item difficulty estimate the most?

I have a student with a known ability level and a set of items with an estimated difficulty. The probability of a student getting an item correct follows an 1 parameter logistic model (also known as ...
user avatar
  • 6,126
0 votes
1 answer
24 views

Interpret ANOVA analysis of model fit IRT, TAM Package

I posted this question on Stack Overflow, but was advised to post it here. I fitted the same data set to a 1PL item response theory model (called onePL below), and ...
user avatar
0 votes
0 answers
42 views

mirt: GRM with symmetrical thresholds

I'm looking for a way to fit a graded response model where the thresholds are symmetrically located around zero. The mirt.model command allows you to add an ...
user avatar
0 votes
0 answers
34 views

How to compare responses of two independent samples of a Likert scale using IRT?

I want to acknowledge that I am very beginner in this topic of Likert scales and Item Response Theory. So, I have responses from two groups to a questionary with 8 items which ordinal responses can ...
user avatar
1 vote
1 answer
28 views

Psychometric test validation question

We are trying to validate a test for young children using IRT analyses (rasch/2pl). Models do not converge due to a floor effect. Teachers also took this assessment. Would it skew results if i ...
user avatar
  • 11
0 votes
0 answers
19 views

IRT: Estimating item difficulty/discrimination for each factor within a second order factor

I'm interested in looking at item difficulty and discrimination for all items within a measurement model which has 5 first-order factors and one second-order factor. I want to be able to compare all ...
user avatar
0 votes
0 answers
9 views

Determine section weights for Norm Development in IRT calibrated Item bank

I have an item bank containing 60 binary items (fitted to IRT 2PL model), divided into 3 sections (easy, medium and difficult) based on b parameter values (less than -.8 is easy, between -.8 and 0.8 ...
user avatar
  • 1
0 votes
0 answers
56 views

How to handle missing data (when items not present) in IRT analysis in R

I am trying to calibrate an Item bank using IRT 2PL analysis in R. I have 60 dichotomous Items in my bank, while at any given instance a random set of 25 items are presented to a test taker. I have ...
user avatar
  • 1
4 votes
2 answers
47 views

Finding a Common Thread in Disparate Indicators

All of the theory I know about factor analysis, scales, etc revolves around separating out indicators into groups that each have (ideally) one underlying dimension. I have a student for whom we'd ...
user avatar
  • 661
1 vote
0 answers
39 views

simulation of an item response theory 2PL model [closed]

I would like to simulate a 2PL probit model. I want to set up some arbitrary item parameters $a=(a_1,...,a_J)$, $b=(b_1,...,b_J)$ and abilities $\theta=(\theta_1,...,\theta_I)$ ($J$ number of items ...
user avatar
  • 53
0 votes
0 answers
46 views

Consequences of inserting non-monotonic items with a dip in the trace line in tests using IRT 3PL

I identified 3 items with curves like this in a 45-question test that uses item response theory with 3 parameters to assess students ability. The plot show the model's prediction (line) and the ...
user avatar
0 votes
0 answers
61 views

Maximal/Minimal Difference Between Output Probabilities Under the 2PL Model

The output probability of the IRT 2PL model is defined as $$P(Y=1|\theta,a,b)=\frac{e^{a(\theta-b)}}{1+e^{a(\theta-b)}}$$ for $\theta,b\in\mathbb{R}$ and $a>0$. $\theta$ is our independent variable ...
user avatar
  • 2,351
0 votes
1 answer
63 views

Scaling formula (reference required)

I came across a very good scaling formula to scale theta based on Item Response Theory to the scale of 31-68. The original post is here. The formula looks like this: $f(x) = \frac{(b-a)(x-min)}{max-...
user avatar
2 votes
1 answer
64 views

Standard 2PL IRT Formulation

The standard 2PL IRT model takes the following form $$P(X = 1\ \big|\ \theta, a, b) = \frac{e^{a(\theta - b)}}{1 + e^{a(\theta - b)}}$$ where $b$ represents the "item difficulty" and informs ...
user avatar
1 vote
1 answer
30 views

What exactly is "requirements for fundamental measurement" for rasch model?

In this introduction of Rasch Model, it says "the Rasch model emphasizes the primacy of the requirements for fundamental measurement". What exactly does "requirements for fundamental ...
user avatar
  • 539
1 vote
0 answers
24 views

Factor Analysis on binary data (dimensionality check before IRT) [duplicate]

I'm trying to do a factorial analysis of binary data (hit / miss) related to the results of a test that consisted of two types of items. My intention is to confirm the number of dimensions to later ...
user avatar
0 votes
0 answers
19 views

IRT models for repetitive items

I'm looking for literature about IRT models that fit for tests in which items are presented for each examinee more than once and/or different questions are asked for the same item. For example, in the ...
user avatar
  • 1
0 votes
1 answer
62 views

Are ability estimates given by ltm and mirt Z-scores?

Are the ability estimates given by fscores in mirt and factor.scores in ...
user avatar
  • 327
1 vote
1 answer
205 views

Estimating ability with IRT when item parameters are known using R package mirt

I have a paper which has utilised IRT on a revised scale of paranoia (R-GPTS). The scale comprises 10 items and there are 5 response options: In the above highlighted responses, this scores 3 / 40. ...
user avatar
  • 213
0 votes
1 answer
145 views

IRT analysis: duplicate columns?

I am trying to create a 2-parameter logistic regression model. I have no NA in my data (233 rows (plus 1 header row) and 458 columns of 0's and 1's only, but many columns are duplicate). I am able to ...
user avatar
  • 1
1 vote
1 answer
49 views

Which IRT model is recommended to capture manifest continuous indicators (items)?

The test under investigation comprises of continuous items, whose response space is not limited to a special interval (such as [0;1]). Instead, my items are developed for measuring the execution speed ...
user avatar
0 votes
1 answer
86 views

There is a easy way in R to check dimensionality in IRT with principal component analysis on standardized residuals?

I suspect that a test done by thousands of students has a dimensional problem (there are two or more dimensions influencing the students' chance of getting the items right). I have a dichotomous ...
user avatar
1 vote
1 answer
36 views

Assess overall question difficulty in Graded Response Model

I used graded response modelling (GRM) to assess the difficulty of items from a survey using a Likert scale from 1 to 5. The results show the difficulty of each item (i.e., 1-5), for each question. I ...
user avatar
  • 11
0 votes
1 answer
28 views

Is the mean of predicted theta scores supposed to be 0?

I predicted theta scores using a GPCM (based on 7-point attitude answers). I expected the mean of the predicted theta scores to be 0. However, it is not. Can anyone explain to me whether the mean of ...
user avatar
  • 1
1 vote
1 answer
92 views

Estimating IRT item parameters when ability and response are know

I have 500,000 students dichotomy responses for a 45 item test. I know the ability of the this students, that this ability was calculated from this responses using 3PL model by Expected a Posteriori (...
user avatar
1 vote
1 answer
62 views

Theta (thetaEst) estimates in CatR with known item parameters - assessing a whole dataset

To my R angels, After a recent voyage of discovery, I have been using IRT to develop a set of GRM derived item parameters on a large dataset of 40000 questionnaire responses. I would like to use these ...
user avatar
1 vote
1 answer
137 views

How do I appropriately examine the dimensionality of binary data?

I have 72 binary variables and, at a theoretical level, I am trying to identify groups of variables that seem to vary together. In practice, I am struggling with how to analyze this data properly. I ...
user avatar
1 vote
1 answer
76 views

Bayesian Multiple Item Response Models?

I am trying to extend the ideas of item response theory to multiple responses. Consider a marketing survey, which asks customers, "what's the deciding factor in whether or not you purchase ...
user avatar
  • 1,700
2 votes
0 answers
149 views

How to incorporate multiple likelihoods in a probabilistic graphical model with Stan?

Data composition: In beta testing of a video game, users were assigned tasks in a many-to-many relationship. At the end of every day, users were asked to self evaluate (for each task) whether they ...
user avatar
  • 1,700
6 votes
1 answer
67 views

What is the role of non-anchor items when equating scales across studies using IRT?

I'm trying to use IRT to equate scales across 5 datasets. Three of them use the same scale (A); the other ones use different ones (B and C) There are items that overlap between A and C, A and B, but ...
user avatar
  • 331
2 votes
0 answers
76 views

Are there supposed to be bounds on parameters in 2PL Item Response Theory models?

Recently I've been studying Item Response Theory (IRT) and have come across some issues with the application side of it. I currently have a dataset of ~200 respondents x 7405 questions (quite ...
user avatar
1 vote
1 answer
86 views

How many parameters are estimated in a 2PL model?

I'm familiar with 1PL IRT models, where you have some data matrix, and you model the latent factors, theta (the trait/ability) and (item) difficulty. ...
user avatar
  • 1,700
4 votes
1 answer
200 views

Do 1PL IRT models measure both ability and difficulty, or just difficulty?

I'm trying to better understand Item response Theory (IRT) from a Bayesian perspective. Hypothetically, suppose I want to use a 1PL model and my data is a binary matrix ...
user avatar
  • 1,700
1 vote
1 answer
59 views

Intuitions behind item response theory?

I'm relatively new to Item Response Theory. After having read some materials on 1PL and 2PL, I have a few thoughts and questions. Say you have a questionnaire that a social psychologist will complete ...
user avatar
  • 1,700
0 votes
1 answer
180 views

IRT: What is enough information? And can a test provide too much?

I was wondering what makes for a good test information value in IRT, and came across this: Reliability: Item and test information functions graphically reflect how reliably the individual items and ...
user avatar
  • 327
2 votes
0 answers
147 views

How to best summarize Likert data (to use as an independent variable in a regression)?

I want to run a regression where one of the explanatory variables is a "summary" (details below) of a set of questionnaire questions that are answered on a Likert scale (although there are ...
user avatar
  • 21
1 vote
1 answer
439 views

Data With Ordinal Responses: Calculate ICC & Assess Model Fit

I have a data set of animal responses with repeated measures. The responses are on a 9-point scale. Here, I present an example data set that is very similar to mine: ...
user avatar
0 votes
1 answer
219 views

Calculating Theta scores for identified subjects (using R package ltm) [closed]

I am using the R package ltm to calculate a 2PL IRT model. goal: creating a list of subjects IDs with the calculated Theta scores. problem: ltm requires that the input includes only item scores, and ...
user avatar
  • 13
0 votes
0 answers
152 views

Testing the fit of polychoric correlation

I’m using polychoric correlations for my work. According to Kampen and Weeren (2017): In order to prevent this questionable research practice, it is recommended that in applications of the ...
user avatar
0 votes
0 answers
270 views

Difference between CFA using WLSMV estimator and IRT

I am trying to understand the differences (or similarities) between conducting CFA on categorical data using the WLSMV estimator in SEM and item response theory analysis. I would appreciate any input. ...
user avatar
2 votes
1 answer
54 views

item response theory - criteria for selecting items

I'm trying to use IRT to select the best items for a one-factor scale that has 20 items. I was wondering what I need to be looking at if I wanted to select 5 or so items from these items (for a short ...
user avatar
  • 331
7 votes
1 answer
614 views

Analyzing a partially crossed design

I have a data set from a test (see below). The scoring algorithm gives each item (item_id) a score (y) that is continuous from $...
user avatar
0 votes
1 answer
150 views

Item response theory (IRT) for continuous responses

I hope this message finds everyone safe and well. I want to estimate the Rasch item difficulty parameters for my test items (dataset below). However, I have two challenges: (1) Item scores are ...
user avatar
0 votes
0 answers
15 views

DIF: validity of carrying out a t-test on the final sample?

I have run a DIF analysis of a questionnaire, to see how autistic men and women answer differently, with a total sample of 530 autistic men and 530 autistic women. I have a general hypothesis that the ...
user avatar
0 votes
0 answers
20 views

DIF: match sample randomly or by age?

I am carrying out a DIF analysis comparing the scores of men and women on a questionnaire. A preliminary ANOVA revealed that when comparing total scores on the test, there is a significant effect of ...
user avatar
0 votes
0 answers
50 views

Wright-Map with multiple values for one item

To give some context: I'm quite new to quantitative research in social sciences and learned about IRT, Rasch Reliability and Wright-Maps yesterday. Today a colleague sent me this image: and asked ...
user avatar
  • 113

1
2 3 4 5