Timeline for What is the Gold Standard for Evaluating the Posterior of a Bayesian Regression Model?
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 11 at 19:51 | comment | added | picky_porpoise | @profPlum No I don't think so. I (finally) posted an answer with details and references. | |
| Aug 28 at 17:04 | comment | added | profPlum | @picky_porpoise Do you mean energy-based (e.g. machine) learning? | |
| Jul 21 at 10:32 | comment | added | picky_porpoise | @profPlum Stephan Kolassa is right in that proper scoring rules asses distributions, not samples. However, there are scoring rules for which we can easily estimate their value based on a sample. I think the energy score is one of them, but currently I have no time to post more details. | |
| Jul 14 at 20:51 | vote | accept | profPlum | ||
| Jul 14 at 18:48 | comment | added | profPlum | Good point it's just a CLT approximation... But I'm not sure what else to do? | |
| Jul 14 at 17:24 | comment | added | jbowman | That is not so! That is an asymptotic result, not a finite sample result! | |
| Jul 14 at 16:48 | comment | added | profPlum | It's ok. I just learned in my case of mean-field variational inference for deep learning, you can apparently prove the distribution of posterior predictions (model outputs) will be gaussian which solves the problem for me. | |
| Jul 14 at 12:39 | comment | added | Stephan Kolassa | You have a point. In principle, proper scoring rules will by definition be minimized in expectation by the correct distribution... which will likely translate into some asymptotic statement if all we have is samples from our predictive distribution. No, I don't know of any work in this direction, sorry! | |
| Jul 13 at 22:11 | comment | added | profPlum | Thanks! But I'm concerned that you mention it's the log-likelihood. Does this method work if you only have samples of the posterior rather than an explicit function (e.g. MCMC)? I'm concerned because high dimensional pdf estimation is tricky (e.g. KDE doesn't scale well). | |
| Jul 13 at 21:00 | history | answered | Stephan Kolassa | CC BY-SA 4.0 |