Timeline for How to see how good my probability estimations are?
Current License: CC BY-SA 3.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 15, 2017 at 6:52 | comment | added | Stephan Kolassa | Sorry for taking a while. I think your last comment is looking at things the wrong way: scoring rules don't compare a single predictive distribution across multiple future true distributions (rows in the table I added) - rather, they allow you to compare different predictive distriibutions for a single true future distribution (columns in the table). After all, that's what we are interested in: to find the best predictive distribution for a future series of outcomes, not to find one predictive distribution that fits widely different future distributions. Does that help? | |
| May 15, 2017 at 6:50 | history | edited | Stephan Kolassa | CC BY-SA 3.0 |
added reference and example
|
| May 11, 2017 at 22:39 | comment | added | Nenko | I have read into the different scoring rules and so far I don't really "like" any of them. For example Brier and logaritmic score rules. If I assign 50% probability to 100 events in a row, whether I get a.100 0's, b. 100 1's or c. 50 0's and 50 1's, all 3 options will give me the same score, while obviously I made good predictions in option c. and terrible ones in a. and b. So scores for predictions of 50% and the surrounding region seem useless. Do you have any idea of a scoring rule that addresses this issue? | |
| May 11, 2017 at 21:07 | vote | accept | Nenko | ||
| May 11, 2017 at 20:49 | history | answered | Stephan Kolassa | CC BY-SA 3.0 |