The Item Response Theory framework encompasses a wide range of psychometric methods and mathematical models (latent trait models) to study individual responses to items defining a measurement instrument (multi-item scales such as questionnaires or tests) that is used to assess a specific construct ...

learn more… | top users | synonyms (1)

0
votes
0answers
11 views

Does it make sense for pairwise correlation to be uniformly higher than an ability?

I am fitting a graded response model to analyze a 16 item, 5 ordered response category scale. It shows univariate properties in an EFA with one very strong principal component. The distribution of ...
0
votes
0answers
15 views

How do you interpret/feedback irt theta scores?

In CTT we would say something like: Your score is higher than 80% of people in the normgroup. How do we interpret/feedback a theta score of 0.85? I know that it is supposed to be 'item-referenced' as ...
1
vote
1answer
32 views

raising the probability of correct answers of nextItem (catR)

Using the catR package I am trying to create a nextItem call that respondents would answer correctly about 65% of the time (even if that means the method is not as efficient as it could be). Children ...
1
vote
0answers
43 views

How should I calculate performance with 3 kinds of response (yes, no and not applicable)?

I am having trouble calculating employee performance. I have an observation form with 34 behaviours, but not all of the behaviours are observed at one given time because it depends on the topic being ...
0
votes
0answers
12 views

Estimating optimal number of items for adaptive testing

I am trying to find out what would be the optimal fixed length of a computer adaptive IRT scored ability test. I have the 2PL parameters of the items. I think the right question to ask is: How to ...
1
vote
1answer
39 views

Dealing with Extreme Response Style

My survey data includes a battery of 36 Likert-scale questions, which are the IVs of interest in my analysis. Respondents rank how "alike" they are to an individual being described in each item. ...
0
votes
0answers
20 views

calculate mirt model with some fixed item parameters

I would like to calculate a 2pl mirt model with parameters for some items fixed. They come from a previous model. ...
0
votes
0answers
17 views

Comparing rasch.mml2 estimates to Winsteps

With the same set of item responses I get different item parameters using Winsteps and the "rasch.mml2" function in the 'sirt' package in R. The difference is about -.61 logits, such that rasch.mml2 ...
0
votes
0answers
16 views

Distractor analysis in language test

The items on my test contain variable number of distractors, and I would like to evaluate the quality of those distractors based on how often they get picked by students. Ideally, I would like to ...
1
vote
0answers
16 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave ...
1
vote
1answer
33 views

Compare covariance ratio

I am using a nonparametric item response theory method called Mokken scale analysis. One of the coefficients that is calculated is called the item scalability coefficient, and it is calculated as the ...
0
votes
0answers
53 views

How can I estimate the personal parameter (theta) with IRT in R?

I would like to obtain the individual personal ability (theta) estimate under 1PL and 2PL IRT model. I have already the item responses of every participants, how can I do this using the R package? I ...
0
votes
0answers
8 views

runtime calculation of irt person-fit statistics for single person

I currently have a large (500+ items) 2PL model (in the form of item parameters) that I use for irt adaptive testing with catR. Each person do a small subset of these items cc.40 items. Its easy to ...
0
votes
1answer
67 views

Detecting Differential Item Functioning (DIF) using ordered logistic regression

I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF. The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. ...
2
votes
1answer
54 views

Is empirical reliability in IRT the counterpart of internal consistency in CTT?

In IRT it is not meaningful to compute internal consistency coefficients such as Cronbachs alpha, because standard error of measurement (and thus measurement precision) is not assumed uniform across ...
0
votes
0answers
40 views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
0
votes
0answers
73 views

How to compare theta estimates and its standard errors to true test score in IRT?

I'm working on analysis of a multiple choice test by means of Item Response Theory (IRT) 3-parameter model. I've got the estimates for theta and its standard errors for each examinee. I want to make ...
0
votes
0answers
62 views

Different results in estimating 3PL model parameters in IRTPRO and mirt(R)

Can someone explain why I obtain different a, b, and c parameters estimates when I use IRTPRO3 and the package mirt from R? The method both uses are Bock and Aiken (1981) - IRTPRO calls BAEM and R ...
2
votes
1answer
60 views

Meaning of large values of parameter b (|b| >>) 4 in IRT theory

I'm working with a set of data from a 100 itens, 5 alternatives, medicine test and analyzing them by means of IRT (. The IRT - 3 parameter model yielded some values of b that are very large (...
1
vote
1answer
77 views

how do you estimate irt parameters if Theta and response is given?

I have good Theta estimates for people (from an already existing test). These people have also answered a few (2-5) untried experimental items. How can I get the a,b params for these new items given ...
3
votes
1answer
104 views

How to interpret meaning of regressors in this logistic regression model?

I'm trying to understand the model in this paper where they treat the item response theory model as a form of logistic regression. In the model the probability of getting an item (question) correct ...
1
vote
1answer
23 views

Can the test information function be used for a field test item bank?

I am field testing a few forms of items (with embedded anchor items) and will be analyzing the items using IRT, a 2PL model. These items will likely be used to create an item bank that can then create ...
2
votes
1answer
108 views

MCMC Estimation of Multidimensional 2PL IRT Model Using JAGS

I'm trying to prepare for some more advanced work involving MIRT models I'll be doing later this year by fitting a very simple multidimensional 2PL model to some simulated data using MCMC methods in ...
0
votes
0answers
39 views

Raschmodel on the single dimension as modeltest for mirt?

For unidimensional dichotomous IRT-Models testing the Rasch Model is a necessary requirement. What about applying multdimensional IRT-Models? Are there any other informationscources except AIC and ...
0
votes
1answer
29 views

Internal consistency and other item analyses

Is it appropriate to run internal consistency and other item analyses on a test that has questions with different response formats. To be clearer, all questions have a likert response, but some are 5 ...
5
votes
1answer
56 views

BIC in Item Response Theory Models: Using log(N) vs log(N*I) as a weight

In IRT software packages and in the literature it is common to calculate the BIC as $$ \mathrm{BIC} = -2 \cdot \mathrm{logLik} + \log(N)\mathrm{Npars} $$ where $N$ is the number of rows in wide ...
1
vote
1answer
65 views

Questions regarding interpretation of item parameters: inconsistent results between classical item analysis and item response analysis

I was not trained as a psychometrician, so my questions may be concept-related. If you know different resources, books, or articles that can clear my confusions, please let me know. Thank you very ...
3
votes
1answer
106 views

Is it possible to do IRT distractor analysis using R?

I've been using the mirt package to analyse a few response matrixes I have through IRT. However, every time I need to perform distractor analysis, I end up having ...
4
votes
1answer
219 views

Questions Regarding Item Response Models – the 3PL model vs. the mixture of 2PL and 3PL model

I currently work on an item analysis for an assessment. The assessment consists of 25 multiple-choice questions. Each question has 4 choices with one correct answer. I used different IRT models to ...
2
votes
1answer
88 views

Item discrimination of poorly targeted items

In test theory, it appears to be widely recognized that very "easy" or "very difficult" items, relative to the ability of the sample of respondents, will have deflated correlations with the rest of ...
1
vote
0answers
27 views

How to calculate the parameter value for a test with Item Response Theory?

Given a set of responses to a test with multiple choice I wish to analyse it with Item Response Theory: wikipedia on IRT. I am planing to use the 3PL (3 parameters) in the Item response function. How ...
0
votes
0answers
29 views

Is a representative sample necessary to assess construct validity?

To generalize results to a greater population, a representative sample is necessary. In developing a psychometrically valid instrument, via confirmatory factor analysis or item response theory, we are ...
0
votes
1answer
148 views

Misfitting of drift rate in Wiener model

In a binary-decision experiment, the accuracy and response time of some subjects were recorded. My goal is to fit a Wiener model to response time and accuracy data. The binary experiment has three ...
0
votes
0answers
21 views

Extracting latent factor from ranking data

I want to construct a model to link between a set of quantitative engineering parameters (e.g. Sm, Ra) of 9 material samples to a set subjective qualitative rankings of 6 perceived attributes, e.g. ...
1
vote
0answers
50 views

Recode Binary to scale Variables

I'm looking to find information to recode a set of dichotomous response variables to preferable a 1-5 scale. I have several response variables which when combined with PCA clearly depict certain ...
1
vote
1answer
370 views

What is the statistic to calculate Cronbach's alpha for Polytomous items in R

Does anyone know, what is the function which can calculate Cronbach's alpha for polytomous items with different categories in R? Thank you
0
votes
0answers
118 views

Itemfit to IRT model with missing data

I have matrix of dichotomus correct/false answers with many random missing data. (The data comes from an ability test where questions were randomly drawn from an item bank.) I am trying to find out ...
1
vote
1answer
54 views

Cut Score Determination in IRT

Cut score or cut point is a minimum achievement or ability that examinees should get in order to get a 'Pass'. In CTT, MCQ scoring in the form of percent-correct scores are applied. Let say we ...
0
votes
0answers
31 views

Validity in IRT Style

From my reading, I found that IRT generally provides more sophisticated information. Therefore, in regards of unidimensional 3PL IRT model, how should I report the finding about validity in ...
1
vote
1answer
39 views

Reading recommendations for psychometric refinement

a research project turned up a large number of survey instruments on a specific topic. There is a great deal of conceptual item overlap in these instruments, but they do differ in some key ways, ...
0
votes
1answer
70 views

Comparisons of confirmatory and exploratory model in IRT

I am trying to decide whether a theoretically derived (i.e. confirmatory) IRT model fits the data better than some parsimonious (i.e. exploratory) IRT model. Specifically, I have 14 binary indicators ...
2
votes
0answers
48 views

Likert scale survey student response independence

We administer a confidence survey (likert scale) comprising of 30 items and have conducted PCA to examine dimensionality, item independence and the like. We also conduct student interviews to assess ...
4
votes
2answers
858 views

Reliability in IRT Style

Reporting 'reliability' in IRT is more to 'informations' obtained and a bit different from a classical way using such as Cronbach alpha and KR20. Therefore, how to report the test reliability in ...
1
vote
0answers
31 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
0
votes
0answers
35 views

As a rule of thumb, difference in amount of administered items: classically scored vs adaptive (irt-based) tests?

I am thinking of developing a new adaptive test. One of the main goals is to shorten existing, classically scored tests. Is there a rule of thumb about how many items are needed to administered ...
1
vote
2answers
73 views

Understanding and applying Item Response Theory

In the process of designing a quiz application that can assess the student for understanding of a particular concept, I came across Item Response Theory. I have absolutely no clue whether applying ...
2
votes
0answers
56 views

Are there local maxima in the standard 2-pl model or is there proof that there aren't?

I'm looking to use iterative logistic regression to find the parameter values of the 2 pl model: $ P = \sigma(\alpha_{i} (\theta_{s} - \beta_{i})) $ where $ \sigma(x) = \frac{1}{1+e^{-x}} $ I know ...
1
vote
2answers
139 views

Latent Class Analysis vs Rasch Analysis

I was wondering if anybody could explain the core, philosophical/conceptual differences between Latent Class Analysis and Rasch Analysis. I'm not necessarily referring to the exact mathematical ...
1
vote
1answer
42 views

The probability of a student getting a given question correct and the associated uncertainty (Item response theory)

I'm doing some statistics on multiple choice test scores (with item response theory, (http://en.wikipedia.org/wiki/Item_response_theory). I want to know the probability that a student with a certain ...
6
votes
2answers
890 views

Item Response Theory vs Confirmatory Factor Analysis

I was wondering what the core, meaningful differences are between Item Response Theory and Confirmatory Factor Analysis. I understand that there are differences in the calculations (focusing more on ...