For assessing the fit of a model in a Bayesian framework, 'posterior predictive p-values' (PPPs) are often used. Here, a value close to 0.5 indicates a good fit; a value close to 0 or 1 indicates a poor fit. This is because the PPP is the proportion of time during an MCMC run that a chosen test statistic, generated from a distribution predicted by the model, is higher than the test statistic generated from the distribution of the actual input data. If it is 0.5, the model's predictions are close to what the actual data are.

Is there a way in which a PPP can be calculated for a multinomial logistic regression? Say there are 5 possible (unordered) outcomes (such as different states of the weather in Wisconsin), and we use various predictors (such as the season, time of day, etc.) to generate a multinomial logistic regression model to predict how the probability of each outcome changes at different values of the predictor variables. Once the model has been fitted to the data, we want to know if it is a good fit or not. I have not come across it, but is there a suitable method for obtaining a PPP in this case, or does the 'multinomialness' of the problem make it difficult to obtain? I just want to calculate how well my multinomial logistic regression output actually predicts the data.

  • $\begingroup$ (One potential answer might be to name an appropriate test statistic to use for multinomial data.) $\endgroup$ – Sprog Feb 23 '19 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.