Timeline for What is "Multinomial Deviance" in the glmnet package?
Current License: CC BY-SA 3.0
21 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 10, 2021 at 17:32 | comment | added | David | @JoséBayoánSantiagoCalderón, in your example you are calculating the log likelihood incorrectly! The multinomial deviance calculation is -2*sum(log(u[cbind(1:length(y), y)])) where the log likelihood = sum(log(u[cbind(1:length(y), y)])) | |
| May 10, 2021 at 17:16 | comment | added | David | that's a really nice detailed answer, but simply put the multinomial deviance is equal to -2*[log likelihood] | |
| Mar 20, 2019 at 16:04 | comment | added | José Bayoán Santiago Calderón | I am trying to use the formula for computing the deviance of a probability model, but not being able to obtain the expected results. A minimal working example is set.seed(0); m <- 1000; df <- data.frame(y = ordered(sample(x = c('A','B','C','D','E'), size = m, replace = TRUE)), x = rnorm(m)); library(MASS); m <- polr(formula = y ~ x, data = df); y <- model.response(m\$model)); u <- fitted(m); sum(log(u)) | |
| Jan 21, 2016 at 8:21 | comment | added | max | Wouldn't deviance and log-loss be different if there is noise, that is if the y's are random even after conditioning on x's? | |
| Dec 16, 2015 at 23:53 | history | edited | dsaxton | CC BY-SA 3.0 |
deleted 5 characters in body
|
| Dec 16, 2015 at 19:26 | comment | added | Zach | Thank you so much. I have a much better understanding of this topic now. | |
| Dec 16, 2015 at 19:25 | history | bounty ended | Zach | ||
| Dec 16, 2015 at 15:45 | comment | added | dsaxton | @Zach The empirical log loss is $- n^{-1} \sum_{i=1}^{n} \log [ \hat{p}_{j_i}(x_i) ]$ (it's an estimate of the cross-entropy between the true and estimated models: en.wikipedia.org/wiki/Cross_entropy), which is just the deviance times $1 / 2n$. It should be the same for any classification problem where you have estimated probabilities. | |
| Dec 16, 2015 at 15:34 | comment | added | Zach | Thanks for such an excellent answer! When you say "which is just the empirical log loss multiplied by a constant", what is the constant? Is it always the same, or does it vary problem by problem? Mentally I'm trying to figure out an easy way to convert the "multinomial deviance" scale to "multiclass logloss", which I have a much better intuitive understanding about. | |
| Dec 16, 2015 at 14:27 | history | edited | dsaxton | CC BY-SA 3.0 |
added 112 characters in body
|
| Dec 16, 2015 at 1:20 | history | edited | dsaxton | CC BY-SA 3.0 |
deleted 4 characters in body
|
| Dec 16, 2015 at 1:10 | history | edited | dsaxton | CC BY-SA 3.0 |
added 3 characters in body
|
| Dec 16, 2015 at 0:47 | comment | added | dsaxton | @Zach Sure, I added a bit about log loss. | |
| Dec 16, 2015 at 0:46 | history | edited | dsaxton | CC BY-SA 3.0 |
added 395 characters in body
|
| Dec 16, 2015 at 0:08 | comment | added | Zach | Thank you for the excellent, detailed answer. One last question— how does this deviance function (which I think glmnet computes as "predictive" deviance on out-of-sample data) relate to "multi-class" logloss? | |
| Dec 16, 2015 at 0:07 | vote | accept | Zach | ||
| Dec 15, 2015 at 19:46 | history | edited | dsaxton | CC BY-SA 3.0 |
deleted 144 characters in body
|
| Dec 15, 2015 at 19:40 | history | edited | dsaxton | CC BY-SA 3.0 |
deleted 135 characters in body
|
| Dec 15, 2015 at 19:23 | history | edited | dsaxton | CC BY-SA 3.0 |
deleted 27 characters in body
|
| Dec 15, 2015 at 18:59 | history | edited | dsaxton | CC BY-SA 3.0 |
deleted 1 character in body
|
| Dec 15, 2015 at 18:54 | history | answered | dsaxton | CC BY-SA 3.0 |