Timeline for What is the Gold Standard for Evaluating the Posterior of a Bayesian Regression Model?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 24 at 11:37 | history | edited | Martin Modrák | CC BY-SA 4.0 |
added 1 character in body
|
| Jul 16 at 20:09 | comment | added | profPlum | Ok I'll take a look thanks! | |
| Jul 16 at 4:29 | history | edited | Martin Modrák | CC BY-SA 4.0 |
added 58 characters in body
|
| Jul 16 at 4:25 | comment | added | Martin Modrák | I really encourage you to read the linked Bayesian analysis paper on SBC by me. I've spent a lit of time thinking through those things. Kernel estimates are generally a mess and rarely give you more than what you get by directly working with samples. However, using the posterior density as a test quantity in SBC turns out quite powerful (for reasons discussed in the paper) | |
| Jul 15 at 19:06 | comment | added | profPlum | The problem I had is that probably the "most correct way" (maybe Yao's approach?), would be to estimate the density of the predicted distribution then sort them by density instead of distance. But the only way I could think to do this is with KDE which apparently doesn't scale very well to high dimensions... | |
| Jul 15 at 19:03 | comment | added | profPlum | Omg that's so cool! I literally came up with that idea on my own. It's awesome to know that other people are using it too. I tried using the Lemos approach, but ya I'm not sure ofc that it is 100% valid. | |
| Jul 15 at 5:22 | comment | added | Martin Modrák | And yes, SBC validates all credible intervals simultaneously - the idea is that you look at the rank of the true (simulated) value within the posterior. The distribution of ranks is uniform if and only if all credible intervals are calibrated. | |
| Jul 15 at 5:03 | comment | added | Martin Modrák | The easiest (and typical) way is to look at various univariate projections/functions (test quantities) one at a time. The approach by Yao is a clever way to look at the full joint, but may require much more computation. The Lemos way is to choose a distance metric(s) which once again make the problem 1D. I think, but so far cannot prove that the "distance metrics" in Lemos approach actually correspond 1-to-1 to "test quantities" in SBC. | |
| Jul 14 at 21:28 | comment | added | profPlum | Thanks for the variety of options. Regarding SBC is the purpose to validate all credible intervals simultaneously? I had an idea about how to approximate this but it's confusing in high dimensions what the best approach is. | |
| Jul 14 at 8:35 | history | edited | Martin Modrák | CC BY-SA 4.0 |
added 298 characters in body
|
| Jul 14 at 8:30 | history | answered | Martin Modrák | CC BY-SA 4.0 |