0
$\begingroup$

I posted this question on Stack Overflow, but was advised to post it here.

I fitted the same data set to a 1PL item response theory model (called onePL below), and then to a 2PL item response theory model (called twoPL) below.. Subsequently, I ran an ANOVA analysis to investigate which model had the best fit (using anova(onePL, twoPL). I used the TAM package. Is anybody here familiar with interperting the output of such an ANOVA analysis? In the attached picture it looks like model 1 had a better fit, but surely, that's incorrect?

I would be very grateful for any help or suggestions.

Model loglike Deviance Npars AIC BIC Chisq df p
onePL 4499.997 8999.994 7 9013.994 9050.36 46.96485 5 0
twoPL 4476.514 8953.029 12 8977.029 9039.371 NA NA NA
Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

looks like your data has 6 items. In the onePL model you estimate 6 difficulty parameters and one variance parameter of the latent trait distribution, in total 7 parameters.

In the twoPL model you estimate 6 difficulty and 6 discrimination parameters (location and variance of the latent trait distribution is fixed), in total 12 parameters.

In the anova the test statistics is the differences between the deviances (which are $-2*log(likelihood)$; noted in the column Deviance). Note that the entries in the loglike-columns should be negative (check your results). The differences between the deviances is 46.965. Now check this against the $\chi^2$ distribution with 5 degrees of freedom (differences in the number of parameters; df). Using 1-pchisq(8999.994-8953.029, 5) (which results in the p column value) can be interpreted as the probability that the differences in the model fit (i.e., deviances) is due to chance, given 5 more parameters.

Note that AIC and BIC (lower is better) agree that the 5 additional parameters help improving the model fit.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Dear @Tom, many thanks for your answer! It certainly clarifies things. I guess I was confused by the layout of the table - to me it looked like the onePL was significantly better given that the p-value was reported on that line. (In the original output the loglike values were indeed negative (a copy error of mine). Again: thanks! $\endgroup$
    – profUlph
    Commented Mar 16, 2022 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.