The data set which I am trying to analyse is Student Test Data.

I have a data of responses (either 1/correct response or 0/incorrect response) on some questions of a set of students. I have fitted a "3 parameter logistic model" for each question and then calculated the goodness of fit estimate and hence the respective p-values.

Now my problem is I don't know how to aggregate these individual goodness of fit estimates to get a total goodness of fit for the whole model, Is there any measure (preferably in the programming language R) which can suggest about the whole model depending on the p-values of multiple tests. The package which I used is ltm inside R.

  • $\begingroup$ Unfortunately I am not able to help you with that but I tried to make the title more explicit to have a better chance to attract the right people. Do you think the new title is OK? $\endgroup$ – Gala Jun 18 '13 at 6:39
  • $\begingroup$ Possibly relevant to this issue: stats.stackexchange.com/questions/95785/… $\endgroup$ – philchalmers May 23 '14 at 20:36

I think it would be best to fit a single model for all the items e.g. mod <- tpm(response_matrix, IRT.param = TRUE) where response_matrix has a column for each item and a row for each person with a 1 or 0 in each cell. You can then obtain the AIC and BIC for the model using summary(mod) (see under Model Summary) and if you need each individually, summary(mod)$AIC and summary(mod)$BIC.

  • $\begingroup$ My reading of the question is that the OP does not have a single model but one for each question. $\endgroup$ – mdewey Feb 9 '18 at 15:16
  • $\begingroup$ Thanks @mdewey, I've rephrased my answer to recommend a single model instead. IMO this makes more sense because it allows you to compare between items as well as people. $\endgroup$ – Greg Feb 10 '18 at 19:22
  • 1
    $\begingroup$ AIC and BIC are not goodness of fit statistics, nor is their values in isolation all that meaningful. They are only useful in nested comparison situations, which is not what OP was asking. $\endgroup$ – philchalmers Feb 20 '18 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.