How to Assess Goodness of Fit of Multinomial Logit Model for Ungrouped Data?

Question

I have some data which starts like:

     died white los age admission
1101    0     1   8   2  Elective
317     1     1  13   4  Elective
623     1     1  16   7  Elective
1295    1     1   1   5    Urgent
1006    1     1   1   9  Elective
1197    0     1   5   6    Urgent
203     0     1   3   3  Elective
1317    1     1  21   5    Urgent
338     0     1  26   6  Elective
1478    1     1   2   4 Emergency

I am modelling admission on died, white, los and age using a multinomial logit model.

Does anybody have know how I could assess the goodness of fit of the model? Perhaps there is a hypothesis test I could use? Or a plot I could create?

If the data was grouped, I could use a $\chi^2$ goodness of fit test, but I don't know what I could use for ungrouped data. Neither do I know of any informative plots I could create.

Thanks for any suggestions.

Answer 1

One approach would be to carry out cross-validation (CV; or some other data-partitioning method) during model fitting. If you are using R, this can be done with the caret package.

Essentially this process (in the case of CV) the data is partitioned into training and test sets. First the model is fitted to a subset of the data, the training set) and then it is used to see how well it predicts the test set data. This is done iteratively over every partitioned and used to create a prediction error metric, which gives you an idea of the specificity and sensitivity of the model. for classification data such as this it will output a confusion matrix (which aren't confusing as the name might suggest).

As for assumption checks of model fit for logistic and multinomial models - its a bit of dark art as there aren't any simple ones. The one approach is to calculate rho and check and see if the dispersion of the data is roughly equivalent to that which is predicted by the underlying distribution.

I think the above assessment with error metrics is a little more straight forward though and gives you a good idea of model performance, particularly whether it is appropriate for new data.

I hope this helps! I will add an edit if I think of anything else though.

EDIT: another approach might be to fit a proportional log-odds model potentially which may be easier to assess in some ways.