3
$\begingroup$

I was wondering whether it is possible to calculate the Brier score for multinomial models in R?

For example:

library(nnet)
data(iris)
mlogit <- multinom(Species ~ ., data = iris)

and given the definition of the Brier score for multinomial model on Wikipedia. I do not believe I can compute the Brier score, because I lack the class probabilities of the non-realized classes for each observation right? Or am I missing something?

$\endgroup$
3
$\begingroup$

doi:10.1002/sim.5321 may give you some ideas. I don't think an overall Brier score for polytomous outcomes has been fully worked out. But at least you could report $k-1$ Brier scores for predicting the probability of $k-1$ out of the $k$ levels of $Y$.

$\endgroup$
3
  • $\begingroup$ Dear Frank; A bit of a long shot I am sure but I wonder if you still have any of these references to hand please, given that citeUlike is no more. $\endgroup$ – user20650 May 17 at 13:51
  • $\begingroup$ Fixed above - thanks. $\endgroup$ – Frank Harrell May 17 at 20:00
  • $\begingroup$ Great, thank you. $\endgroup$ – user20650 May 17 at 22:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.