# Interpret ANOVA analysis of model fit IRT, TAM Package

I posted this question on Stack Overflow, but was advised to post it here.

I fitted the same data set to a 1PL item response theory model (called onePL below), and then to a 2PL item response theory model (called twoPL) below.. Subsequently, I ran an ANOVA analysis to investigate which model had the best fit (using anova(onePL, twoPL). I used the TAM package. Is anybody here familiar with interperting the output of such an ANOVA analysis? In the attached picture it looks like model 1 had a better fit, but surely, that's incorrect?

I would be very grateful for any help or suggestions.

Model loglike Deviance Npars AIC BIC Chisq df p
onePL 4499.997 8999.994 7 9013.994 9050.36 46.96485 5 0
twoPL 4476.514 8953.029 12 8977.029 9039.371 NA NA NA

looks like your data has 6 items. In the onePL model you estimate 6 difficulty parameters and one variance parameter of the latent trait distribution, in total 7 parameters.
In the twoPL model you estimate 6 difficulty and 6 discrimination parameters (location and variance of the latent trait distribution is fixed), in total 12 parameters.
In the anova the test statistics is the differences between the deviances (which are $$-2*log(likelihood)$$; noted in the column Deviance). Note that the entries in the loglike-columns should be negative (check your results). The differences between the deviances is 46.965. Now check this against the $$\chi^2$$ distribution with 5 degrees of freedom (differences in the number of parameters; df). Using 1-pchisq(8999.994-8953.029, 5) (which results in the p column value) can be interpreted as the probability that the differences in the model fit (i.e., deviances) is due to chance, given 5 more parameters.
Note that AIC and BIC (lower is better) agree that the 5 additional parameters help improving the model fit.
• Dear @Tom, many thanks for your answer! It certainly clarifies things. I guess I was confused by the layout of the table - to me it looked like the onePL was significantly better given that the p-value was reported on that line. (In the original output the loglike values were indeed negative (a copy error of mine). Again: thanks! Mar 16, 2022 at 8:47