Hot answers tagged

19

A good way to evaluate the quality of software is to perform simulations with known population parameters and observe how well these values can be recovered. Better yet, comparing parameter recovery to other known software is also a good idea since then you will have a general idea of what's happening if there are peculiarities in the results. In my ...


17

In some ways you are right, CFA and IRT are cut from the same cloth. But it many ways they are quite different as well. CFA, or more appropriately item CFA, is an adaption of the structural equation/covariance modeling framework to account for a specific type of covariation between categorical items. IRT is more directly about modeling categorical variable ...


11

Have a look at Section 1.6 ("The linear regression perspective") in De Boeck and Wilson (2008) Explanatory Item Response Models (http://www.springer.com/de/book/9780387402758) and Formann, A. K. (2007), (Almost) Equivalence between conditional and mixture maximum likelihood estimates for some models of the Rasch type, In M. von Davier & C. H. Carstensen (...


9

The accepted answer does not give models where the response is bounded between [0,1]. There are IRT models for exactly the case where the response variable is continuous, but bounded in this way. For example, Samejima [1] describes exactly this case. These models are sometimes called Continuous Response Models (CRM), and the case they're addressing is ...


8

R is widely used in scientific circles for published papers. R stores your data in RAM, so either it will be able to process your data set or it won't -- depending on whether the data and processing fit in memory -- there is no degraded mode where you get results but they are less accurate. (Technically, there are packages that let you work with larger data ...


8

As far as I can tell you are describing a partially crossed design. The good news is that this is one of Doug Bates's main development goals for lme4: efficiently fitting large, partially crossed linear mixed models. Disclaimer: I don't know that much about Rasch models nor how close a partially nested model like this gets to it: from a brief glance at ...


7

@Philchalmers answer is on point, and if you want a reference from one of the leaders in the field, Muthen (creator of Mplus), here you go: (Edited to include direct quote) An MPlus user asks: I am trying to describe and illustrate current similarities and differences between binary CFA and IRT for my thesis. The default estimation method in Mplus ...


6

In general this is a bad idea. ltm uses MML estimation, hence zero response vectors are still used in estimation and give information about the 'difficulty' of an item, unlike joint ML is which case these need to be removed (like in the program LOGIT). The same thing can be said about removing rows with missing data. While it shouldn't introduce bias if ...


6

The Rasch model is a specific model under a very large umbrella term that is Item Response Theory. See for example the wiki on IRT which specifies the Rasch model as the 1 parameter IRT model


6

A dynamic program will make short work of this. Suppose we administer all questions to the students and then randomly select a subset $\mathcal{I}$ of $k=10$ out of all $n=100$ questions. Let's define a random variable $X_i$ to compare the two students on question $i:$ set it to $1$ if student A is correct and student B not, $-1$ if student B is correct ...


5

As I stated in the comments above, missing data can be handled by either the ltm or mirt package when the data is MCAR. Here is an example of how to use both on a dataset with missing values: > library(ltm) > library(mirt > set.seed(1234) > dat <- expand.table(LSAT7) > dat[sample(1:(nrow(dat)*ncol(dat)), 150)] <- NA > head(dat) ...


5

I can't find a way to do this in the ltm package, though it's relatively straightforward if you are willing to use the mirt package. First, write out some arbitrary matrix or data frame consisting of possible but random response patterns. You can include the actual response patterns you are interested in as well for later use. dat <- matrix(sample(c(...


5

If you have a continuous indicator, then you would use factor analysis. Think of FA as linear regression and IRT it's logistic regression brother.


5

Looks like I'm quite late to the game here, but the mirt package package can estimate WLE scores for dichotomous and polytomous models. You start by fitting, say, a graded response model to your data (or whatever your model may be, PCM, generalised PCM, nominal, rating scale, etc; see ?mirt for the possible options) then compute either a table summary of the ...


5

In Classical Test Theory observed test scores $X$ could be defined as: $$X = T + E$$ where $T$ are the true scores and $E$ is an error of measurement. This means that their variance is: $$\sigma^2_X = \sigma^2_T + \sigma^2_E$$ In this case, reliability could be defined as: $$ \rho_{xx'} = \frac{\sigma^2_T}{\sigma^2_X} = \frac{\sigma^2_X - \sigma^2_E}{\...


5

If you already have established your cut score, and cannot set it again, then the only thing I can think of doing is mapping the range of $\theta$ values that match the observed scores corresponding to your cut score. You will then have to figure out which specific $\theta$ value will be the official cut score, it may make sense to choose the lowest value in ...


5

As far as I understand, standard alpha() function from the psych package (http://personality-project.org/r/html/alpha.html) is not appropriate here, but I'm not absolutely sure about that. It seems to me that, in order to calculate Cronbach's $\alpha$ in R for mixed data, including polytomous items, you can use either function scoreItems() from the psych ...


5

It's the factor.scores function: WIRStheta <- ltm::factor.scores(two_pl)


4

I think I figured out the issue (after much pain): If any of the item columns contains less than two question responses (i.e. the rest are NA), then the library will throw the above error. The solution was to drop any columns (items) that didn't meet that criteria.


4

You can compute test information curves from your IRT parameter estimates. These curves give you the precision of the test at each $\theta$ of the latent trait. The information $I$ can be transformed into the standard error of estimate $SEE$, which is a direct estimate of the reliability of the test at that $\theta$: $SEE = 1 / \sqrt{I}$. The metric of ...


4

No, it is not a bad idea. However, when you look at the posterior distributions for $\alpha_{p,w}$ you will see you haven't learned all that much about those $p,w$ combinations for which you have no data; they'll still be centered at 0, with a spread that's determined largely by the $p,w$ combinations for which you do have data (and the rest by your prior). ...


4

I've included a simdata() function in the mirt package in R for calculating simulated IRT data given a variety of know conditions for several different classes of uni- and multidimensional IRT models. So if you need something a little more flexible that may be a good place to look, and should save you from having to reinvent the wheel.


4

I don't fully understand question 1; IRT is a whole area which I used to know a little about but haven't looked at in a decade. But, in any case, adding scales that represent different things would certainly violate unidimensionality, regardless of whether it is IRT or classical test theory. But that is something that you can assess. Regarding your second ...


4

fit <- rasch(LSAT) ## Factor scores for all subjects in the ## original dataset LSAT factor.scores(fit, resp.patterns = LSAT)


4

The ltm package in R does not have effective global model fit statistics available; one could construct the $G^2$ statistic from the complete-data tables in this package, but this statistic behaves extremely poorly in moderate to large sized tests due to extreme levels of data sparseness. However, the mirt package does have suitable limited information tests ...


4

As a precursor, the IRT approach to this problem is very demanding computationally due to the higher dimensionality. It may be worthwhile to look into structural equation modeling (SEM) alternatives using the WLSMV estimator for ordinal data since I imagine less issues will exist. Plus, including external covariates is much easier within that framework. Both ...


4

To help you get started with IRT, consider getting Baker's book. You can also get a free download of his software. It is a self-directed learning tool and you might find it quite useful to start your understanding of this field.


4

To start off, let's look at what we mean by reliability. Reliability is often thought of as how consistent a measure will be across different measurement scenarios, with everything being equal except the occasion (same assessment, same conditions, same people, different day, e.g.). Reliability can also be thought of as the ability to distinguish between two ...


4

To start, the Rasch model doesn't estimate any slope parameter in the model, and in fact treats them all as fixed (so called Tau-equivalent). Depending on how you parametrise the model will make this more apparent (i.e., fix all the discriminations to 1 and free the latent variance, or fix the latent variance to 1 and freely estimate the discriminations ...


4

Of course, and I have found this quite useful in the past. Simply fit something flexible that captures information from each category, like the nominal response model, to the items of interest, and inspect the response curves graphically or by looking at the scoring/ordering coefficients and intercepts. That will give you an indication of how the distractors ...


Only top voted, non community-wiki answers of a minimum length are eligible